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EXECUTIVE  SUMMARY 


Volume  5  of  the  Report  of  the  Defence  and  Civil  Institute  of  Environmental  Medicine/Air 
Transport  Group  (DCIEM/ATG)  study  team  describes  the  development  of  a  proposed  new 
training  programme  devoted  to  Human  Factors  in  Decision  Making  (HFDM).  It  is  envisaged 
that  the  HFDM  training  syllabus  would  replace  existing  Aircrew  Co-ordination  Training  (ACT) 
within  the  CC-130  community.  The  proposed  training  can  be  distinguished  from  other 
approaches  with  similar  goals  (either  explicit  or  implicit)  by  its  base  within  a  theoretical 
framework  of  human  information  processing.  The  differences  lie  less  in  the  content  than  in  the 
way  the  material  is  organised  and  shaped  by  theory. 

The  proposed  HFDM  training  is  based  on  two  related  models  of  the  human  information 
processor.  They  are. 

The  Information  Processing  (IP)  Model.  The  IP  model  claims  that  all  factors  that 
determine  operator  workload  can  be  reduced  to  their  effect  on  the  amount  of 
information  that  has  to  be  processed,  or  their  effect  on  the  time  available  before 
a  decision  has  to  be  implemented.  Under  the  assertion  that  humans  are  limited  in 
the  rate  at  which  they  can  process  information,  operator  workload,  performance 
and  error  production  can  all  be  shown  to  be  functions  of  the  time  pressure  or  the 
ratio  of  time  to  process  the  information  to  the  time  available. 

The  Perceptual  Control  Theory  (PCT)  Model.  The  PCT  model  is  based  on  the 
assertion  that  all  living  organisms  operate  as  a  multi-layered  closed  loop  control 
system.  The  set  points  for  the  control  loops  are  the  organism’s  goals  (or  how  you 
want  to  see  the  state  of  the  world).  PCT  asserts  that  all  observable  behaviours  are 
intended  to  operate  on  the  external  world  so  that  the  perceived  world  state 
matches  a  desired  state  (the  goal).  This  model  provides  a  coherent  framework  for 
integrating  the  concepts  of  workload,  situation  awareness,  mental  models  and 
decision  making  performance.  It  establishes  the  absolute  necessity  of  feedback  in 
goal  directed  activity. 

The  models  are  complementary,  as  the  IP  model  sits  within  the  PCT  framework.  Together  they 
integrate  much  of  what  is  known  about  human  information  processing  and  decision  making. 

Topics  for  HFDM  training  come  directly  from  this  theoretical  framework.  The  proposed 
syllabus  is  made  up  of  1 1  related  modules  covering  the  following  topics:  an  introduction  to  the 
decision  making  loop;  leadership  and  followership;  the  emergence  of  communication  and 
captaincy  in  teams  and  groups;  sensation/perception;  goal  setting;  action  selection;  management 
of  control,  attention,  time  and  knowledge;  and  finally  communication.  Each  module  is  anchored 
in  the  theoretical  framework  and  its  effect  on  decision  making.  The  core  knowledge  and  skill  set 
that  are  expected  to  come  from  each  module  are  described  for  the  purposes  of  assessing  the 
efficacy  of  training. 

A  clear  distinction  has  been  made  between  the  knowledge  development  phase  of  training  and 
skill  development.  In  the  proposed  training  programme,  skill  development  would  be  done  in  a 
hands-on  team  environment  using  a  variety  of  situations  such  as:  role  playing  and  team 
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exercises;  case  study  analysis;  low  and  high  fidelity  simulation;  instructional  rides;  and  on-the- 
job-training  (OJT).  It  is  at  this  level  that  there  exists  the  greatest  potential  for  tailoring  the 
training  to  the  trainee’s  position  on  the  aircraft,  and  making  the  training  relevant  to  each 
individual’s  stage  of  development  (upgrade  training,  recurrency  training,  etc.). 

A  programme  such  as  the  one  outlined  in  this  Volume  requires  the  following  additions  to  the 
current  CC-130  training  commitment: 

•  1  extra  classroom  day  added  to  the  current  ACT  course; 

•  2  extra  days  devoted  to  hands-on  team  decision  making  training  for  all  positions  (as 
preparation  for  simulator  sessions  in  the  case  of  the  Aircraft  Commanders  [ACs],  Co-pilots 
[CPs]  and  Flight  Engineers  [FEs]); 

•  2  days  per  year  of  HFDM  refresher  training  within  the  Squadrons  (this  could  be  in 
conjunction  with  a  safety  stand-down);  and 

•  3  extra  days  for  all  those  going  through  AC  upgrade  training  or  re-qualification  training. 

Manning  levels  within  the  training  Squadron,  to  accomplish  this  task,  have  not  been  assessed. 
Overall  these  are  modest  additions  to  a  training  programme  that  is  currently  devoted  almost 
entirely  to  technical  issues.  Considering  that  human  factors  (HF)  failures  are  assigned  to 
somewhere  between  70%  and  80%  of  the  accidents  and  incidents  around  the  world,  the  proposed 
amount  of  time  spent  in  HF  training  is  not  excessive. 
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INTRODUCTION 


This  volume  is  the  fifth  in  a  six  part  series  describing  a  human  factors  study  for  the  Canadian 
Forces  CC-130  aircraft  (see  References  [1-5]  for  details  of  the  remaining  documents  in  this 
series).  This  volume  deals  with  the  issue  of  Crew  Resource  Management  (CRM),  specifically  as 
it  relates  to  decision  making  in  multi-crew  cockpits. 

CRM  programmes,  were  bom  in  the  aftermath  of  the  1978  United  Airlines  (Flight  173)  accident 
at  Portland,  Oregon,  although  the  ideas  had  been  around  for  some  time  prior  to  that.  In  the 
succeeding  years,  as  CRM  has  matured,  it  sometimes  seems  to  take  on  a  chameleon  character  as 
it  reflects  an  ever  increasing  domain  of  interest.  It  has  reached  the  point  where  some  people  use 
the  term  CRM  training  and  human  factors  training  synonymously.  As  the  specifics  of  individual 
CRM  programmes  differ  from  one  another,  it  is  essential  to  look  at  the  core  topics  covered  and 
the  overall  training  objectives  before  drawing  conclusions  about  the  effectiveness  of  a  specific 
programme.  Robert  Helmreich,  of  the  University  of  Texas’  Aerospace  Research  Project,  argues 
that  4  generations  of  CRM  programmes  have  been  implemented  to  date  [6].  He  characterises 
them  as  follows: 

•  Generation  1  (1981-1986) 

—  derived  from  corporate  management  development  training 

—  focus  on  individual  management  style/interpersonal  skills 

—  intended  to  fix  the  ‘Wrong  Stuff  captains  with  a  subsidiary  goal  of  making  First 
Officers/Co-pilots  (FOs/CPs)  more  assertive; 

•  Generation  2  (1986-present) 

—  more  team  based 

•  team  building 

—  focus  on  concepts 

•  situation  awareness 

•  stress  management 

—  modular 

•  error  chain 

•  individual  decision  making  (DM)  models; 

•  Generation  3  (1986-present) 

—  systems  approach 

—  focus  on  specific  skills/behaviours 

—  integration  with  technical  performance 

—  emphasis  on  evaluating  human  factors 

—  special  training  for  check  airmen/instructors 
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—  broadened  perspective 

•  include  flight  attendants,  dispatchers  and  maintenance; 

•  Generation  4  (1994-present) 

—  performance  data  guide  training 

—  integration  of  CRM  into  technical  training 

—  proceduralisation  of  CRM 

•  CRM  aspects  added  to  checklists 

—  specialised  curriculum  topics 

•  automation,  etc. 

—  Evaluation  of  human  factors  in  full  mission  simulation. 

Helmreich  and  his  colleagues  have  recently  argued  for  a  further  change  in  focus,  one  that  is 
intended  to  return  CRM  training  to  its  original  goal  of  error  management  [6].  Helmreich  et  al. 
present  the  following  recipe  for  a  fifth  generation  CRM. 

•  Generation  5  (199?  -  future) 

—  Focus  on  managing  human  error 

—  Training  in  the  limitations  of  human  performance 

•  ubiquity  of  human  error 

•  use  of  accident  and  incident  data  to  illustrate  these  limitations 

—  Continuation  of  earlier  generation  training  topics  under  an  error  management  format. 

Over  these  years,  CRM  has  gained  widespread  acceptance  amongst  civilian  and  military 
organisations  alike.  All  major  western  operators  of  multi-place  aircraft  have  some  sort  of  CRM 
programme  in  place.  Many  certifying  bodies  mandate  CRM  training.  Considering  the  dearth  of 
hard  evidence  showing  the  benefits  of  CRM  training,  the  enthusiasm  for  these  programmes  may 
be  perceived  to  be  based  largely  on  faith.  This  situation  has  fuelled  an  often  lively  debate, 
exacerbated  by  the  current  climate  of  fiscal  restraint  and  the  requirement  to  justify  any  resources 
devoted  to  safety-related  programmes.  While  few  would  argue  that  safety  is  not  important,  the 
question  “...how  much  must  be  spent  on  safety...”  is  a  legitimate  question  to  ask  in  what  is 
otherwise  an  open-ended  situation.  Unless  CRM  is  delivering  expected  improvements  in  safe 
flight  performance,  dollars  spent  on  CRM  might  be  spent  more  effectively  elsewhere  (more 
conventional  training  time,  etc.).  This  is  not  to  say  that  CRM  is  only  a  safety  related  issue.  An 
argument  can  be  made  to  show  that  CRM  potentially  increases  both  the  efficiency  and  safety  of 
an  operation.  In  fact  the  United  States  Air  Force  (USAF)  puts  “...Maximise  operational 
effectiveness  and  combat  capability...”  as  number  one  in  a  list  of  three  objectives  for  their  CRM 
programme  [7]. 

Alan  Unit  (in  1978  he  was  the  National  Transportation  Safety  Board  [NTSB]  Human  Factors 
Group  Co-Chairman  for  the  UA  173  accident,  now  the  Technical  Advisor  for  Human 
Performance  in  the  USAF  Safety  Agency)  quoted  a  1997  study  by  Jensen  and  Benal  for  the  US 
Department  of  Transportation/Federal  Aviation  Authority  (DoT/FAA)  in  which  the  authors 
claimed  that  all  aircrew  errors  could  be  classified  according  to  three  major  categories  [8],  namely 

•  procedural  errors; 

•  perceptual  motor  errors;  and 
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•  decisional  errors. 


Diehl  used  this  classification  to  analyse  a  selection  of  US  general  aviation  accidents,  airline  and 
USAF  Class  A  mishaps.  For  each  class  of  aviation,  decisional  errors  were  implicated  in  over 
half  of  the  accidents.  Therefore,  to  the  extent  that  CRM  programmes  emphasise  decision 
making,  there  is  an  expectation  that  they  will  have  a  positive  impact  on  safety.  However,  for  the 
purposes  of  interpreting  the  following  studies,  it  cannot  be  assumed  that  CRM  and  Aeronautical 
Decision  Making  (ADM)  are  identical. 

As  a  long  time  advocate  of  both  CRM  and  ADM  programmes,  Diehl  addresses  the  evidence  for 
the  benefits  of  this  type  of  training.  He  points  to  six  world-wide  government  sponsored 
evaluations  of  the  efficacy  of  ADM  training  (1  Australian,  2  Canadian  and  3  US).  Diehl 
describes  these  studies  as  examining  errors  made  during  “...short,  seemingly  routine,  cross 
country  observational  flights.”  Specially  trained  observers  placed  subjects  in  a  series  of  decision 
making  situations  and,  it  is  claimed,  surreptitiously  recorded  judgmental  errors.  These 
experiments  were  double  blind  (the  observers  did  not  know  who  had  been  given  ADM  training 
and  the  subjects  did  not  know  the  true  purpose  of  the  experiment).  Those  subjects  who  had 
received  ADM  training  performed,  on  average,  better  than  those  who  had  not.  Mean  average 
error  rates  were  reduced  by  a  factor  that  ranged  from  8%  to  46%.  In  general  those  programmes 
that  included  ADM  in  simulator  and  flight  training  did  better  than  those  based  on  manuals  and 
lectures  alone.  Each  of  these  studies  dealt  with  the  general  aviation  fleet,  but  used  quite  rigorous 
methods  that  give  credence  to  the  results  obtained. 

Diehl  also  describes  a  number  of  longitudinal  studies  of:  major  rotorcraft  organisations;  the  US 
Navy;  and  Air  Force  Airlift  Command  (MAC).  It  is  difficult  to  draw  conclusions  from 
longitudinal  studies  as  many  factors  might  change  over  time  in  addition  to  the  variable  of  interest 
(in  this  case  the  provision  of  ADM  or  CRM  training).  However,  Diehl  claims  that  the  one  major 
difference  between  MAC  training  and  that  of  other  commands  in  the  two  five-year  periods 
investigated  (1981-1985  for  the  before  data;  1986-1990  for  the  after  data)  was  the  addition  of  a 
CRM  programme.  If  the  differences  in  these  studies  can  really  be  attributed  to  ADM/CRM 
training  the  numbers  are  impressive,  with  accident  rates  reducing  by  a  factor  of  28  to  81%. 

We  are  now  entering  an  era  where  it  is  almost  certain  that  crews  involved  in  accidents  will  have 
been  exposed  to  some  sort  of  CRM  training.  This  doesn’t  mean  that  CRM  is  necessarily  failing 
as  nobody  has  claimed  CRM  training  would  eliminate  all  human  factors  related  accidents.  Yet 
many  argue  that  CRM  must  involve  more  than  awareness  training  if  it  is  to  have  the  expected 
effects.  This  quote  from  the  accident  investigation  board  of  AA965  near  Cali,  Columbia  in  1995 
makes  the  point  most  poignantly  [9]. 

“...This  accident  demonstrates  that  merely  informing  crews  of  the  hazards  of  over 
reliance  on  automation  and  advising  them  to  turn  off  the  automation  is 
insufficient  and  may  not  affect  pilot  procedures  when  it  is  needed  most. 

This  accident  also  demonstrates  that  even  superior  CRM  programs,  as  evidenced 
at  AA,  cannot  assure  that  under  times  of  stress  or  high  workload,  when  it  is  most 
critically  needed,  effective  CRM  will  be  manifest.  In  this  accident,  the  CRM  of  the 
crew  was  deficient  as  neither  pilot  was  able  to  recognize  the  following: 

•  The  use  of  the  FMS  was  confusing  and  did  not  clarify  the  situation. 

•  Neither  understood  the  steps  necessary  to  execute  the  approach,  even  while 
trying  to  execute  it. 

•  Numerous  cues  were  available  that  illustrated  that  the  initial  decision  to 
accept  runway  19  was  ill  advised  and  should  be  changed. 
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•  They  were  encountering  numerous  parallels  with  an  accident  scenario  they 
had  reviewed  in  recent  CRM  training. 

•  The  flight  path  was  not  monitored  for  over  a  minute  just  before  the  accident. 

Although  the  accident  flightcrew  articulated  misgivings  several  times  during  the 
approach,  neither  pilot  displayed  the  objectivity  necessary  to  recognize  that  they 
had  lost  situation  awareness  and  effective  CRM. 

The  FAA  has  encouraged  airlines  to  implement  effective  CRM  programs  and  has 
mandated  it  as  a  fundamental  part  of  the  advanced  qualification  program  (AQP), 
an  innovative  method  of  training  airline  pilots.  The  FAA  has  issued  an  advisory 
circular...,  No.  120-51A,  that  provides  guidance  to  airlines  on  elements  needed 
for  a  [sic]  effective  CRM  program.  The  [advisory  circular]  identifies  topics  that 
should  be  included  in  a  CRM  program.  These  include:  communications  processes 
and  decision  behavior;  briefings;  inquiry/advocacy/assertion,  crew  self-critique; 
conflict  resolution;  communications  and  decision  making;  team  building  and 
maintenance;  and  individual  factors/stress  reduction.  Within  the  topic  of  team 
building,  the  [advisory  circular]  suggests  that  workload  management  and 
situational  awareness  be  addressed,  so  that  '...  the  importance  of  maintaining 
awareness  of  the  operational  environment  and  anticipating  contingencies  ...  ’  is 
addressed. 

Aeronautica  Civil  believes  that  this  accident  demonstrates  the  difficulty  in 
training  for  enhanced  pilot  situational  awareness.  The  crew  of  AA965  was 
trained  in  a  CRM  program  that  adhered  to  the  guidance  of  [advisory  circular] 
120-51A,  and  that  had  added  additional  information  on  hazards  unique  to  the 
South  American  operating  environment.  The  evidence  indicates  that  this  crew 
was  given  background  material  and  information  necessary  to  avoid  this  accident, 
but  during  a  stressful  situation,  when  it  was  most  needed,  the  information  was  not 
applied,  most  likely  because  the  critical  situation  was  not  recognized. 

Offering  further  guidance  on  training  in  situation  awareness  does  not  address  the 
fact  that  pilots  who  have  lost  or  not  achieved  situation  awareness  cannot  be 
expected  to  recognize  that  they  have  lost  or  not  achieved  it.  More  importantly, 
these  pilots  cannot  be  expected  to  develop  a  mechanism  to  efficiently  achieve  it.  ” 

(pp.  46-47) 

The  archives  of  the  CRM-Developers  forum  on  the  internet  (http://www.caar.db.erau.edu/crm/) 
provide  further  evidence  that  not  everyone  is  fully  convinced  that  CRM  training  is  hitting  the 
mark.  Some  contributors  have  made  the  case  that  CRM  (and  human  factors)  has  become  a  buzz 
word  that  has  lost  its  impact  [10].  The  lack  of  a  convincing  theoretical  position  has  only 
muddied  the  waters.  In  a  recent  book  chapter  the  question  “Why  Crew  Resource 
Management?...”  was  asked  [11].  In  the  discussion  that  followed,  the  authors  traced  some  of  the 
history  of  CRM  programmes  within  airline  and  military  operations;  provided  a  number  of  case 
histories  where  crews  apparently  did  not  draw  on  all  the  resources  available  to  them  —  usually 
with  tragic  consequences;  and  presented  some  of  the  experimental  evidence  in  support  of  the 
current  team  training  approach  to  CRM.  While  the  empirical/anecdotal  evidence  is  often 
compelling,  little  is  presented  that  would  qualify  as  providing  a  theoretical  justification  for  CRM. 

This  report  describes  an  approach  to  human  factors  training  with  the  explicit  goal  of  improved 
decision  making.  It  is  proposed  that  training  based  on  this  approach  would  replace  the  current 
Aircrew  Co-ordination  Training  the  CC-130  community.  The  proposed  training  can  be 
distinguished  from  other  approaches  with  similar  goals  (either  explicit  or  implicit)  mainly  by  its 
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base  within  a  theoretical  framework  of  human  information  processing.  The  differences  lie  less  in 
the  content  of  the  course  than  in  the  way  the  material  is  organised  and  shaped  by  theory. 


5 


TRAINING  MODEL 


The  objective  of  training  is  to  establish,  within  each  individual,  a  behaviour  that  did  not  formerly 
exist,  to  re-enforce  a  behaviour  that  is  already  present,  or  to  align  an  existing  behaviour  with 
some  target  behaviour.  This  process  usually  follows  a  progression  which  starts  with  a  general 
awareness  of  the  topic  and  ends  with  a  set  of  target  behaviours  that  are  robust  and  reliable.  The 
model  of  learning  that  has  guided  the  development  of  the  Human  Factors  in  Decision  Making 
(HFDM)  package,  presented  in  this  Volume,  can  be  represented  as  follows  [12]: 


theory  —» through  instruction/case  studies/examples  — >  knowledge 
knowledge  — »  through  exercises/role  plays  — >  skills 

skills  — >  through  organisational  reinforcement  — >  everyday  practice. 


Rasmussen  [13]  argues  that  human  decision  making  is  based  on  three  types  of  problem  solving 
strategies.  He  terms  these  strategies  knowledge-based,  rule-based,  and  skill-based  (K-R-S). 
Knowledge-based  problem  solving  is  the  most  information  processing  intensive  strategy  of  the 
three.  It  involves  deductive  reasoning,  what  is  sometimes  called  algorithmic  problem  solving, 
and  makes  heavy  demands  on  working  memory  (a  recognised  bottle-neck  in  human  information 
processing).  Knowledge-based  problem  solving  characterises  our  behaviours  in  the  early  days  of 
any  new  activity.  It  is  also  the  type  of  strategy  we  generally  use  in  unforeseen  or  novel  situations 
(e.g.,  the  crew  of  the  United  DC- 10  Flight  232  at  Sioux  City,  Iowa).  These  circumstances  are 
related  by  the  novelty  of  the  situation.  Knowledge-based  problem  solving  can  be  very  precise, 
but  it  is  also  slow  due  to  the  amount  of  uncertainty  (information)  that  must  be  resolved. 

Less  demanding  at  the  cognitive  level  is  rule-based  problem  solving.  As  the  name  suggests, 
rule-based  problem  solving  can  be  used  when  an  appropriate  behaviour  is  linked  to  an  initiating 
condition  (the  need  to  act)  by  some  form  of  rule  (e.g.,  in  the  form  of  an  IF  this  THEN  that 
statement)  or  rule  of  thumb  (also  called  a  heuristic  [14]).  A  heuristic  is  a  problem  solving 
strategy  that  yields  an  adequate  solution  to  the  problem  in  a  reasonable  amount  of  time  (e.g.,  the 
1  in  60  rule  for  correcting  off-track  navigational  error).  The  information  processing  involved  in 
this  type  of  decision  making  is  less  than  that  for  knowledge-based  problem  solving,  as  it  is 
generally  sufficient  to  link  the  need  to  act  with  one  of  a  small  set  of  rules.  This  type  of  problem 
solving  is  often  less  precise  than  knowledge-based  solutions,  but  is  more  timely.  This  represents 
the  type  of  speed  accuracy  trade-off  that  is  built  into  one  of  the  models  that  forms  the  theoretical 
framework  for  the  HFDM  training  (the  Information  Processing  or  IP  model1). 

Finally  there  is  skill-based  problem  solving,  sometimes  called  automatic  [15,  16].  This 
represents  the  most  refined  level  of  skill  acquisition,  where  stimulus  and  response  appear  to  be 
linked  directly  by  a  process  of  pattern  recognition.  Skill-based  problem  solving  is  the  fastest 
form  of  decision  making,  as  little  working  memory  or  conscious  processing  appears  to  be 

1  The  IP  model  is  described  later  in  this  Volume. 
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involved.  This  type  of  information  processing  strategy  can  be  associated  with  Klein’s 
recognition-primed  (R-primed)  decision  model  [17]. 

A  skilled  aviator  must  be  practised  in  all  of  the  complex  aviation  tasks,  and  will  be  required  to 
exercise  all  three  types  of  decision  making  strategies.  The  ability  to  exercise  each  type  of 
decision  making  strategy  appropriately  can  therefore  be  considered  a  skill  in  its  own  right,  in  the 
sense  of  the  learning  model  presented  at  the  beginning  of  this  Section.  The  role  of  training  is  to 
push  the  decision  making  associated  with  each  target  behaviour  as  far  down  in  Rasmussen’s 

hierarchy  as  is  possible  and  appropriate  (i.e.,  K-»R-»S),  acknowledging  that  the  ability  to  do  this 
will  depend  on  the  activity  being  trained.  Note  that  the  ability  to  generalise  each  strategy 
reduces  as  one  goes  from  K->R->S.  It  is  the  objective  of  the  training  system  that  crews  operate 
largely  at  the  skill-based  level  so  far  as  the  mechanics  of  flying  the  aircraft,  and  operating  all  the 
on-board  equipment  during  routine  operations,  is  concerned.  As  the  situation  departs  from  the 
normal,  rule-  and  knowledge-based  behaviours  will  tend  to  assume  greater  importance. 

The  tenets  of  Ecological  Interface  Design  (EID)  argue  that  equipment  design  should  support 
problem  solving  at  each  of  the  levels  of  Rasmussen’s  hierarchy  [18].  Training  needs  to  provide 
similar  support.  The  ideas  behind  EID  came  largely  from  the  design  of  interfaces  for  the  nuclear 
power  industry  and  are  intended  to  promote  better  decision  making  particularly  in  fault  finding 
activities  and  the  diagnosis  of  abnormal  situations.  Skill  and  rule-based  problem  solving  relies 
on  being  able  to  retrieve  from  memory  the  specifics  of  a  response.  If  memory  fails  there  is 
usually  little  from  which  the  correct  response  can  be  derived  at  this  level.  One  can  either  retrieve 
the  correct  behaviour  or  not.  Having  higher  level  knowledge,  that  allows  the  appropriate 
response  to  be  deduced,  can  compensate  for  this  initial  failure  of  retrieval.  For  example,  in  a 
typical  emergency  undercarriage  extension  procedure,  the  undercarriage  handle  can  be  up  or 
down  and  the  circuit  breaker  can  be  in  or  out.  If  one  practices  this  procedure  many  times  over 
the  appropriate  responses  (put  handle  down,  pull  circuit  breaker)  are  likely  to  exist  at  the  skill- 
based  level.  At  the  next  level  up,  the  response  might  be  coded  in  the  form  of  a  rule  (before 
manual  extension  put  handle  in  the  down  position  and  pull  the  circuit  breaker  —  the  arguments 
down  and  up  have  to  be  correctly  retrieved  from  memory  whenever  the  rule  is  fired).  If  one  has 
learnt  the  causal  relationships  between  likely  undercarriage  failures  and  the  subsequent 
behaviour  of  the  system  then  one  can  deduce  the  appropriate  response  (e.g.,  if  an  intermittent 
electrical  or  hydraulic  failure  has  prevented  the  undercarriage  from  extending  then  the  gear  will 
retract  —  and  override  the  manual  extension  —  if  the  fault  clears  and  the  handle  remains  up  with 
the  circuit  breaker  in;  therefore  the  appropriate  action  is  handle  down  and  circuit  breaker 
pulled).  At  this  higher  level  a  strategy  can  be  deduced  that  will  probably  apply  to  all,  or  at  least 
most,  airframes. 

The  development  of  the  proposed  HFDM  course  syllabus  presented  in  this  Volume  was  guided 
by  the  following  principles: 

Principal  Purpose 


1.  Provide  a  clear  focus  for  the  training  —  in 
this  case  it  is  decision  making. 


•  To  place  all  training  material  in  the  context 
of  a  final  objective. 

•  To  provide  a  bench  mark  for  assessing  the 
relevance  of  all  training  material. 

•  Emphasise  the  product  (decision  making) 
rather  than  the  process. 
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2.  Provide  a  high  level  theoretical  framework 
of  human  performance  and  decision 
making. 


3.  Use  a  multi-tiered  approach  to  training  that 
develops  deeper  knowledge  as  aircrew 
progress  from  initial  training  through  the 
various  phases  of  recurrent  training  and 
upgrades.  Training  should  also  be 
distributed  throughout 

—  ab  initio  training  (eventually) 

—  initial  conversion  training 

—  upgrade  training 

—  recurrent  training 

—  coursework 

—  simulator  and  role  playing 

—  on-the-job-training  (OJT). 

4.  Re-use  the  theoretical  framework  at  all 
stages  of  development. 


5.  Provide  a  consistent  mapping  between 
training  materials  and  the  desired  outcome. 


6.  Provide  examples  of  desirable  behaviours 
rather  than  just  what-not-to-do. 


7.  Support  all  levels  of  the  learning  model  so 
that  theory  promotes  knowledge, 
knowledge  is  turned  into  demonstrable 
skills,  and  these  skills  are  made  everyday 
practice  by  the  imposition  of  organisational 
standards  and  values. 


•  By  constantly  returning  to  the  framework 
learning  will  be  facilitated  by  repetition. 

•  To  provide  a  framework  which  establishes 
the  inter-dependence  between  individual 
course  items. 

•  Establish  a  connection  between  the  product 
and  the  process. 

•  Establishes  causality  which  aids 
knowledge-based  problem  solving. 

•  To  present  the  knowledge  as  required  (just- 
in-time -training)  .  A  new  right-seater  will 
not  need  to  have  the  depth  of  knowledge 
that  a  left-seater  should  have  — 
particularly  at  the  level  of  strategic  decision 
making  (indeed  the  existence  of  OJT 
implies  that  a  new  right-seater  is  an 
apprentice  in  training). 

•  Maintains  the  freshness  and  relevance  of 
recurrent  training. 

•  Provides  undergraduate  through  to  post 
graduate  level  training. 


•  Repetition  promotes  learning. 

•  The  framework  provides  the  scaffolding  on 
which  all  the  individual  knowledge  is  hung. 

•  To  facilitate  retention  of  knowledge  and  the 
retrieval  of  that  knowledge  when  action  is 
required. 

•  To  present  positive  role  models  rather  than 
negative  ones. 

•  Provide  example  behaviours  that  can  be 
modelled. 

•  To  speed  the  acquisition  of  target 
behaviours. 

•  To  achieve  the  highest  possible  level  of 
safe  flight  performance  and  mission 
effectiveness. 
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8.  Required  core  knowledge  should  be 
identified. 

9.  Skill  acquisition  should  be  demonstrated 
through  observable  behaviours. 


•  To  provide  a  means  of  assessment. 

•  To  provide  a  means  of  assessment. 

•  To  provide  a  model  for  students. 
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THEORETICAL  FRAMEWORK 


The  proposed  HFDM  training  is  based  on  two  related  models  of  the  human  information 
processor.  They  are. 

The  Information  Processing  (IP)  Model.  The  IP  model  claims  that  all  factors  that 
determine  operator  workload  can  be  reduced  to  their  effect  on  the  amount  of 
information  that  has  to  be  processed,  or  their  effect  on  the  time  available  before 
a  decision  has  to  be  implemented  [19,  20].  Under  the  assertion  that  humans  are 
limited  in  the  rate  at  which  they  can  process  information,  operator  workload, 
performance  and  error  production  can  all  be  shown  to  be  functions  of  the  time 
pressure  or  the  ratio  of  time  to  process  the  information  to  the  time  available. 

The  Perceptual  Control  Theory  (PCT)  Model.  The  PCT  model  is  based  on  the 
assertion  that  all  living  organisms  operate  as  a  multi-layered  closed  loop  control 
system  [21,  22].  The  set  points  for  the  control  loops  are  the  organism’s  goals  (or 
how  you  want  to  see  the  state  of  the  world).  PCT  asserts  that  all  observable 
behaviours  are  intended  to  operate  on  the  external  world  so  that  the  perceived 
world  state  matches  a  desired  state  (the  goal).  This  model  provides  a  coherent 
framework  for  integrating  the  concepts  of  workload,  situation  awareness,  mental 
models  and  decision  making  performance.  It  establishes  the  absolute  necessity  of 
feedback  in  goal  directed  activity. 

The  models  are  complementary  as  the  IP  model  sits  within  the  PCT  framework.  Together  they 
integrate  much  of  what  is  known  about  human  information  processing  and  decision  making. 

To  put  these  models  into  context,  suppose  we  start  with  something  that  is  familiar  —  a 
household  air-conditioning  system  (see  Figure  1).  Most  air-conditioning  systems  operate  in  a 
closed  loop  fashion,  sensing  the  temperature  of  the  air,  comparing  the  temperature  value  sensed 
with  the  thermostat  set  point,  and  controlling  the  compressor  according  to  the  difference  between 
the  desired  value  and  die  sensed  value.  But  the  compressor  is  capacity  limited  and  can  only 
provide  a  certain  amount  of  cooling  power.  This  fixes  the  rate  at  which  temperature  changes  can 
be  corrected,  as  doors  are  opened  or  shut,  or  pieces  of  heat  producing  equipment  are  turned  on  or 
off.  The  IP/PCT  model  asserts  that  human  decision  making  occurs  in  an  analogous  fashion 
(Figure  1).  The  following  equivalencies  apply. 

Air  Conditioning  System  Human  Decision  Maker 


•  Thermostat  set  point 

•  Temperature  and  humidity  sensors 

•  Valves,  compressor,  heat  exchanger 
properties 

•  Rated  compressor  power  (BTU) 

•  Physical  limits  on  cooling  capacity  (size  of 
compressor,  heat  exchanger,  wear,  etc.) 


•  Goals 

•  Eyes,  ears,  sense  of  smell  and  touch 

•  Mental  models  of  action  selection 

•  Internal  processing  rate  (bits  per  second) 

•  Various  physiological  and  psychological 
stressors  (fatigue,  motivation,  anxiety,  etc.) 
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Figure  1:  Closed  loop  control  of  room  temperature  and  the  IP/PCT  model. 
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Unlike  the  typical  air-conditioner  unit,  the  human  has  many  set  points  (goals)  and  therefore 
many  loops  under  control  at  any  point  in  time.  Also,  the  human  can  be  processing  information 
even  when  there  are  no  observable  behaviours  (thinking,  reasoning,  deducing,  etc.).  Often  multi¬ 
loop  control  is  not  simultaneous  but  relies  on  attention  switching  from  loop  to  loop.  Using  the 
IP/PCT  model  framework  of  Figure  1,  a  number  of  subject  headings  emerge  for  a  HFDM 
syllabus.  For  example: 

•  sensation/perception; 

•  goal  setting; 

•  action  selection; 

•  resource  management; 

—  management  of  control  and  attention; 

—  management  of  time  pressure; 

—  management  of  knowledge; 


Figure  2.  Multiple-controller  systems  of  humans  and  machines  (e.g.,  a  Flight  Management 

System  or  FMS). 
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The  IP/PCT  model  can  be  extended  to  situations  where  multiple  controllers  are  acting  on  the 
same  world  variables.  This  is  the  team  or  group  situation.  Note  that  the  other  team  members 
may  be  human  or  machine  (see  Figure  2).  The  general  principles  remain  the  same  for  human- 
human  and  human-machine  interaction  under  this  framework  —  it  is  just  that  machines  are 
generally  far  more  limited  in  the  numbers  and  types  of  ‘behaviours’  they  exhibit. 

It  should  also  be  stressed  that  the  IP/PCT  model  indicates  that  superior  team  performance  starts 
with  superior  performance  from  each  of  the  individuals  that  compose  the  team,  that  is,  team 
performance  is  built  on  top  of  the  highest  levels  of  individual  skill.  Good  team  processes  might 
compensate  for  a  weak  team  member,  but  will  not  make  a  superior  team.  Further,  the  team 
member(s)  that  has  the  greatest  influence  on  overall  performance  might  be  mission  or  mission 
segment  specific  [23].  However,  as  we  see  in  the  following,  certain  emergent  properties  must  be 
supported  if  the  team  is  to  be  effective  —  individual  skill  is  a  necessary  but  not  sufficient 
requirement  for  superior  team  performance. 

From  what  is  known  about  the  behaviour  of  multiple  controller  systems  and  the  conditions  under 
which  stable  control  can  be  realised,  it  becomes  evident  that  there  are  certain  requirements  for 
the  allocation  of  responsibilities  and  the  calibration  of  all  participant’s  mental  models  and  goal 
states  [24].  As  is  shown  in  the  following  material,  leadership/captaincy  issues  and  the  need  for 
communication  are  properties  that  emerge  from  the  IP/PCT  model  when  multiple  controllers  are 
involved. 


Super  Pilot,  DreamTeam  and  HercCrew 

In  the  following  discussion,  three  hypothetical  types  of  aircrew  are  defined.  They  are 
SuperPilot,  DreamTeam  and  HercCrew.  Many  of  the  properties  ascribed  to  SuperPilot, 
DreamTeam  and  HercCrew  come  from  the  two  theoretical  models  presented  at  the  beginning  of 
this  Section.  The  concept  of  a  SuperPilot  and  the  DreamTeam  came  from  an  attempt  to 
understand  the  fundamental  functions  of  crew  communication  and  Captaincy/Leadership  issues 
in  a  multiple  controller  environment.  It  starts  with  the  premise  of  an  all  capable  individual  and 
traces  the  emergence  of  communication  and  leadership  functions  as  one  moves  to  an  idealised 
team  and  then  the  truths  of  the  real  world.  Although  we  might  automatically  think  of  a  team  in 
purely  human  terms,  note  that  the  team  also  includes  advanced  automation  systems. 

Imagine  an  ideal  flight  deck  that  is  ran  by  a  single  individual  —  the  SuperPilot.  What  does  this 
SuperPilot  do? 

SuperPilot s  are  decision  makers,  establishing  goals,  forming  perceptions  about  the  world, 
comparing  these  perceptions  with  their  goals  and  acting  to  correct  discrepancies.  SuperPilots 
ensure  that  the  most  important  information  processing  loops  are  always  controlled  by 
prioritising.  Results  from  lower  level  loops  are  fed  to  higher  level  loops  by  relatively  high  speed 
neural  communication.  These  pilots  are  super  proficient,  so  their  mental  models  always  contain 
the  knowledge  required  to  form  the  appropriate  actions.  Actions  are  always  consistent  with 
SuperPilot'  %  overall  state  of  knowledge  (what  we  might  call  their  mental  models  and  situation 
awareness)  and  are  goal  driven  with  no  slips,  lapses,  omissions,  etc.  The  only  resource  managed 
is  SuperPilot' s  attentional  resources  —  that  is,  what  loops  are  to  be  controlled,  at  what  level  of 
precision,  at  any  point  of  time.  Resource  management  is  entirely  internal. 

But  SuperPilots  are  rare,  so  the  next  best  thing  is  the  DreamTeam.  DreamTeam  crews  have 
exactly  the  same  life  experiences  and  share  entirely  common  mental  models  and  goals.  Because 
each  team  member  takes  a  little  longer  to  process  information  and  make  the  mechanical  actions 
needed  to  operate  all  the  systems  on  board,  two  members  are  required  on  the  flight  deck. 
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So  what  does  the  DreamTeam  do?  The  functions  are  exactly  the  same  as  for  SuperPilot,  but  now 
there  is  a  requirement  for  communication  between  the  flight  crew.  What  might  this 
communication  be  about? 

Because  these  team  members  share  the  same  goals,  priorities  and  mental  models,  anything 
attended  to  in  the  external  environment  will  be  perceived  identically  by  the  members  of  the 
DreamTeam  —  their  actions  in  all  circumstances  will  be  identical  and  indistinguishable. 
Therefore,  the  DreamTeam  would  communicate  in  order  to: 

•  Allocate  responsibilities  (otherwise  both  members  of  the  DreamTeam  will  be  reaching 
for  the  same  controls  at  the  same  time  in  response  to  changes  in  the  external 
environment).  The  only  way  the  DreamTeam  can  approach  the  performance  of 
SuperPilot  is  by  dividing  the  task  load  and  not  competing  for  control. 

•  Pass  the  results  of  internal  processes  (i.e.,  thinking  without  observable  action)  so  that  the 
internal  mental  models  of  both  team  members  remain  the  same  as  the  mission  unfolds. 

•  Provide  feedback  on  system  states  that  are  not  within  the  common  locus  of  attention 
(because  of  the  division  of  duties,  each  member  of  the  DreamTeam  might  be  attending  to 
different  things). 

•  Direct  attention  i.e.,  inviting  direct  observation  rather  than  using  verbal  descriptions  to 
convey  information. 

The  DreamTeam  will  operate  at  a  slightly  lower  level  of  precision  than  SuperPilot.  This  is  a 
direct  function  of  the  time  lost  in  external  (voice,  gesture,  etc.)  versus  internal  (neural) 
communication  speeds.  The  only  new  functions  for  DreamTeam  are  the  requirements  to  allocate 
responsibilities  and  to  replicate  the  complete  mental  model  in  the  other  crew  member  at  all  times. 
This  suggests  a  Captaincy  or  Leadership  role  for  one  of  the  DreamTeam  members  (because  the 
DreamTeam  are  entirely  in  tune  with  each  other,  each  will  come  up  with  the  same  division  of 
duties  for  the  other  —  this  needs  to  be  resolved!).  There  is  no  need  to  assign  a  principle  decision 
maker  or  leader  as  each  member  will  make  the  same  decisions  given  a  common  state  of 
situational  knowledge.  Each  knows  what  the  other  will  do  in  a  given  situation.  However,  the 
role  of  splitting  the  load  should  be  assigned  to  one  of  the  DreamTeam  crew  in  order  to  avoid  the 
need  for  conflict  resolution. 

But  most  of  our  crews  are  made  up  by  the  HercCrew  variety  of  aviators.  These  crews  come  to 
the  job  with  different  life  experiences  and  some  common  training  for  the  task.  They  have 
different  levels  of  proficiency  as  measured  by  their  state  of  knowledge  (mental  models)  of  the 
environment.  Their  decision  and  action  times  are  such  that  they  cannot  accomplish  all  the  tasks 
to  the  same  level  of  precision  that  SuperPilot  and  the  DreamTeam  can.  Therefore,  they  have  to 
shed  more  tasks  than  their  more  proficient  peers  and  prioritisation  (what  they  choose  to  attend  to) 
assumes  greater  importance. 

Time  management  is  the  critical  issue  at  all  levels  of  proficiency  from  SuperPilot  down  to 
HercCrew.  It  is  a  constant  trade-off  between  speed  and  precision  of  performance.  A  hub 
(principle  decision  maker  or  PDM  —  say  the  Aircraft  Commander  [AC],  Mission  Commander 
[MC],  etc.)  and  spoke  (secondary  decision  makers  or  supporting  crew)  arrangement  of  the  team 
is  appropriate  in  this  situation.  Where  there  is  the  possibility  of  more  than  one  outcome  in  a 
given  situation  someone  has  to  choose.  Pre-assignment  of  this  responsibility  resolves  uncertainty 
and  saves  time.  And  time  is  the  critical  resource. 

The  mental  models  of  the  HercCrew  members  can  not  be  assumed  to  be  identical  and  therefore 
their  decisions/actions  will  not  be  the  same.  The  PDM  must  assume  that  certain  assigned 
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activities  will  be  completed  by  the  team  members.  This  allows  the  PDM  to  delegate 
responsibility  and  switch  attention  to  other  tasks.  Confirmation  of  final  results  may  be  all  that  is 
necessary.  Only  through  this  type  of  off-loading  can  the  PDM  free  up  time  to  control  the  high 
level  loops  that  are  essential  to  the  executive  role  of  the  PDM,  for  example,  strategic  goal  setting, 
and  risk  management.  This  can  not  be  done  at  the  expense  of  overloading  the  secondary  decision 
maker(s),  however,  as  this  can  adversely  effect  the  outcomes  of  the  delegated  tasks!  Note  that 
the  role  assigned  to  the  PDM  is  an  executive  role  of  risk  management,  related  to  the  control  of 
strategic  goals.  It  does  not  undermine  the  delegation  of  tactical  decision  making  roles  to  other 
crewmembers  (e.g.,  in  a  pilot-flying,  pilot-not-flying  situation). 

The  following  communication  functions  for  the  DreamTeam  can  be  added: 

•  Establish  common  goals.  If  team  members  are  trying  to  accomplish  conflicting  goals  the 
system  will  go  unstable  particularly  if  both  are  controlling  identical  variables  in  the 
environment.  This  is  a  fundamental  property  of  multiple  controller  systems. 

•  Build  common  mental  models  of  system  states.  This  pertains  only  to  the  mental  models 
that  each  member  of  the  team  might  call  on  to  control  the  loops  for  which  they  are 
responsible  .  Not  every  one  needs  to  know  everything,  and  in  a  time  pressured  environment 
this  is  a  luxury  that  can  not  be  afforded.  However,  as  time  permits,  the  alignment  of  mental 
models  should  be  a  goal. 

When  there  is  human-human  interaction,  a  secondary  communication  role  emerges  related  to: 


•  Establishing  and  maintaining  effective  external  communications.  This  involves: 

—  establishing  leadership, 

—  establishing  the  level  of  trust, 

—  establishing  the  authority  gradient, 

—  establishing  receptiveness,  attentiveness,  co-operativeness,  assertiveness,  etc. 

Therefore,  the  roles  of  communication  in  a  flight  deck  of  HercCrew  can  be  summarised  as 
follows. 

•  Allocation  of  responsibilities  (AC,  MC). 

•  Establishing  common  goals  (AC,  MC). 

•  Building  common  mental  models  of  system  states  (All). 

•  Providing  feedback  on  system  states  and  directing  attention  (All). 

•  Establishing  and  maintaining  communication  channels  (All). 

From  this  list  of  functions,  together  with  the  theoretical  framework  provided  by  the  IP  and  PCT 
models,  it  is  possible  to  formulate  a  list  of  topics  that  might  be  included  in  a  Human  Factors  in 
Decision  Making  course.  From  these  topics  a  skill  set  can  be  derived  that  will  also  form  the  basis 
for  an  assessment  system.  A  course  structure  might  be. 

•  Introduction  to  the  Decision  Making  Loop. 

•  Leadership  and  Followership. 

•  SuperPilot,  DreamTeam  and  You. 
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•  Sensation/Perception. 

•  Goal  Setting. 

•  Action  Selection. 

•  Resource  Management. 

—  Management  of  Control  and  Attention. 
—  Management  of  Time  Pressure. 

—  Management  of  Knowledge. 

•  Communication. 


Captaincy 

From  the  preceding  discussion.  Captaincy  is  seen  to  be  a  property  emerging  from  the  need  to 
establish  stable  control  in  a  multi-controller  environment,  and  to  exercise  the  functions  of  the 
executive  role  of  primary  decision  maker.  The  concepts  of  Captaincy  and  Leadership  seem  to  go 
together,  but  leadership  is  not  necessarily  isolated  in  the  Captain.  Transport  Canada  promotes 
leadership  as  a  responsibility  for  all  crew  members  [25].  Pettitt  and  Dunlap  [26]  suggest  that 
“...leadership  is  a  general  systematic  and  relational  process  that  emphasises  the  ability  to 
exercise  skill  in  the  movement  towards  goal  achievement.”  They  further  suggest  that 
* ... leadership  is  proactive  rather  than  reactive,  and  necessarily  takes  into  account  other 
members  of  the  group”  But  although  the  term  ‘leadership’  has  been  with  us  fpr  nearly  200  years 
[26],  and  despite  a  huge  literature  on  the  topic,  a  concise,  unambiguous,  definition  has  not  been 
established.  Captaincy  is  best  seen  as  a  role  (this  is  consistent  with  the  SuperPilot  analysis  in  the 
preceding  Section)  with  leadership  as  one  of  the  traits  that  Captains  amongst  others  should 
exhibit  —  what  will  be  unique  to  Captains  are  the  goals  in  which  they  take  leadership. 

The  presence  of  leadership  suggests  the  existence  of  the  complimentary  attribute  of  followership. 
There  should  be  no  stigma  attached  to  the  term  followership  as  it  does  not  imply  passiveness  or 
submissiveness.  Pettitt  and  Dunlap  conclude  [26] 

“followership. ..[is].. .the  ability  to  contribute  to  task  and  goal  accomplishment 
through  supportive  technical,  interpersonal,  and  cognitive  skills.  Followership  is 
not  a  challenge  to  the  Captain’s  authority,  but  neither  is  it  unthinking  compliance 
with  directives,  especially  if  those  directives  endanger  the  safety  of  the  operation. 

Further  our  research  suggests  that  the  concept  of  followership  inflight  operations 
is  in  every  way  similar  to  the  view  advocated  by  Hollander  and  Offerman  (1990): 

‘being  a  follower  can  be  an  active  role  that  holds  the  potential  for 
leadership... behaviors  seen  to  represent  effective  leadership  include  attributes  of 
good followership’  (p.  180)." 

Leadership  and  followership  skills  are  important  contributors  to  overall  crew  effectiveness. 
Pettitt  and  Dunlap  [26]  found  overall  crew  effectiveness  was  generally  higher  when 
leadership/followership  skills  were  present  in  the  Captain  than  in  the  First  Officer.  That  is,  the 
Captain’s  skills  have  a  large  effect  on  overall  crew  effectiveness.  This  parallels  the  results  of  the 
Phase  I  CC-130  study  where  AC’s  behaviours2  were  seen  to  have  a  dominant  effect  on  rated 
crew  proficiency  [27]. 


2  Note  that  in  certain  missions  or  mission  segments,  the  major  influence  on  overall  crew  performance  could  change 
from  crewmember  to  crewmember;  For  example,  while  the  AC  may  well  exert  the  dominant  influence  in  a  strategic 
mission,  the  Navigator  may  assume  a  prime  role  in  a  tactical  mission  etc. 
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Pettitt  and  Dunlap  identify  the  following  6  leadership/followership  skills  and  observable 
behaviours  in  the  civilian  airline  environment. 

Skill  Observable  behaviour 


Envisioning 


Crewmember  develops  and  articulates  a  picture  of  the  future  or 
desired  state. 


Modelling 


Receptiveness 


Influence 


Adaptability 


Crewmember’s  conduct  with  other  employees  and  passengers  is 
consistent  with  the  Company’s  highest  standards. 

Crewmember  gives  attention  to  other  crewmember’s  ideas, 
concerns  or  questions. 

Crewmember  obtains  a  commitment  from  others  to  ideas  or 
actions  using  a  variety  of  interpersonal  skills. 

Crewmember  states  the  need  to  make  adjustments  to  changing 
environments  and  abnormal  situations. 


Initiative 


Crewmember  begins  an  action,  without  external  direction,  to 
respond  to  an  operational  deficiency. 


But  what  are  the  other  attributes  associated  with  the  concept  of  Captaincy  and  how  are  these 
attributes  demonstrated?  It  is  not  enough  to  say  “...we  know  it  when  we  see  it...”  Those  who 
have  it  are  not  the  concern.  It  is  those  who  do  not  currently  demonstrate  adequate  Captaincy 
skills  that  need  guidance  and  development.  For  them,  one  needs  to  present  a  role  model  that  can 
be  imitated. 


In  an  effort  to  define  the  observable  behaviours  associated  with  good  versus  bad  Captaincy 
skills,  a  formal  knowledge  elicitation  procedure  —  the  repertory  grid  —  was  used.  A  contractor 
from  the  study  team  visited  424,  426,  429  and  436  Squadrons  during  the  period  April  to 
September  1997  to  administer  the  procedure.  102  CC-130  crewmembers  (21  ACs,  12  CPs,  18 
Navigators  [NAVs],  22  FEs,  21  Loadmasters  [LMs],  plus  2  Search  and  Rescue  Technicians 
[SARTECHs],  2  Aeromedical  Evacuation  Officers  [AEOs]  +  4  non-completions  due  to  the 
priority  of  duty)  have  participated  in  this  exercise  at  the  time  of  writing  this  report.  For  the 
purposes  of  analysis  the  SARTECHs  and  AEOs  were  absorbed  into  the  LM  group  as  non-cockpit 
crew.  A  detailed  set  of  instructions  for  conducting  the  knowledge  elicitation  process  is  included 
at  Appendix  2  to  this  Volume. 

A  summary  of  the  process  follows.  All  participants  were  advised  that  they  were  free  to  leave  at 
any  time  and  that  their  participation  was  voluntary.  Two  took  this  option.  Each  participant  was 
asked  to  name  the  three  best  Captains  they  knew,  and  the  three  worst.  This  information 
remained  in  the  possession  of  the  respondent  at  all  times,  with  the  contractor  always  unaware  of 
the  Captains  named.  The  sheet  containing  the  names  was  taken  by  the  participant  at  the  end  of 
the  session.  This  was  done  to  ensure  confidentially.  A  mythical  ‘ideal’  AC  was  added  to  the 
pool  of  candidates  making  7  in  all. 

Each  participant  in  the  knowledge  elicitation  process  was  asked  to  describe  how  one  of  a 
randomly  selected  triad  of  the  7  ACs  was  different,  in  terms  of  observable  Captaincy  behaviours, 
from  the  other  two.  They  were  then  asked  how  the  remaining  pair  were  similar,  relative  to  the 
first  selected  AC.  Rating  scales  are  formed  during  this  procedure,  and  each  of  the  7  ACs  (3 
good,  3  less  good  and  the  ideal)  is  placed  on  the  scale  according  to  how  much  of  behaviour  in 
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question  they  display.  A  new  triad  is  formed  and  the  process  repeated  until  the  participants  can 
no  longer  generate  similarities  and  differences  between  members  of  the  triads. 


Only  preliminary  data  analyses  have  been  conducted  so  far.  Detailed  analysis  has  been  held  over 
until  all  data  are  gathered.  Using  a  principle  component  analysis,  the  three  main  factors  that 
appear  to  characterise  each  individual’s  concept  of  Captaincy  were  abstracted  from  the  data  for 
the  first  25  respondents.  Nine  main  categories  of  Captaincy  behaviours  are  emerging.  They  are 
listed  below  with  Pettitt  and  Dunlap’s  leadership/followership  skills  for  comparison. 


CC-130  Study  of  Captaincy 
Behaviours/Skills 

Sees  and  communicates  the  big  picture 
Receptiveness 

Technical  and  procedural  skill 


Pettitt  and  Dunlap’s 
Leadership/Followership  Skills 

Envisioning 

Receptiveness 

(Influence) 


Functions  well  under  pressure  Adaptability 

Can  re-focus  crew’s  attention  and  re-delegate  Adaptability,  Influence 
tasks  in  a  timely  fashion 


Establishes  trust  and  respect 
Professional  conduct 
Crew  development  skills 


Influence 

Modelling 

Initiative 


Attends  to  crew  welfare 


Initiative. 


To  the  original  list  of  9  categories,  3  additions  were  made  as  a  result  of  subsequent  analysis. 

They  are: 

•  sociability 

•  patience 

•  decisiveness  in  decision  making. 

There  are  no  corresponding  skills  in  Pettitt  and  Dunlap’s  list  for  these  additions. 

As  postulated,  there  were  differences  in  the  perception  of  good  Captaincy  behaviours  from  seat 

to  seat.  It  appears  that: 

•  ACs  favour  technical  and  procedural  skill,  functions  well  under  pressure,  and  ability  to  focus 
crew’s  attention  and  delegate  tasks', 

•  CPs  favour  sees  and  communicates  the  big  picture,  functions  well  under  pressure,  and  crew 
development  skill', 

•  FEs  favour  technical  and  procedural  skill,  receptiveness,  and  sociability-, 

•  NAVs  favour  ability  to  focus  crew’s  attention  and  delegate  tasks,  professional  conduct  and 
patience-,  and 
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•  LMs  favour  attends  to  crew  welfare,  receptiveness,  sociability. 

A  more  detailed  analysis  is  in  preparation.  The  information  will  provide  teaching  material  for 
the  HFDM  syllabus  and  guide  an  assessment  package  by  which  Captaincy  behaviours  may  be 
scored.  Because  behaviours  are  generally  observable,  measurement  is  made  easier.  An  external 
observer  (a  trainer  or  Standards  Officer)  can  look  for  and  score  the  presence  or  absence  of  target 
behaviours. 
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PROPOSED  SYLLABUS 


In  Appendix  1  to  this  Volume,  a  draft  syllabus  is  presented  which  is  intended  to  provide  a  plan 
for  the  future.  Eleven  modules  make  up  the  syllabus.  The  topics  are  derived  from  the  theoretical 
basis  presented  previously.  Appendix  1  presents  brief  course  notes  and  a  first  attempt  at 
developing  graphical  representations  which  illustrate  some  of  the  core  concepts.  The  syllabus 
was  developed  to  this  level  of  detail  to  facilitate  discussion  with  8  Wing,  Trenton,  training 
personnel.  It  is  a  prototype  to  be  shaped,  edited,  or  recast  according  to  8  Wing  requirements. 
Sufficient  detail  is  included  to  show  how  such  a  syllabus  can  be  developed  into  a  full  training 
programme  —  it  is  not  currently  complete,  as  this  would  be  inappropriate  until  8  Wing  have  had 
full  input.  Incomplete  skill  sets  are  presented  for  the  modules,  and  examples  of  appropriate 
exercises,  case  studies  and  videos  are  given  for  demonstration  purposes. 

A  summary  of  the  1 1  modules  appears  below. 

Module  1:  Introduction  to  the  Decision  Making  Loop 

Summary:  Accidents  statistics;  majority  of  accidents  attributed  to  human  factors  issues; 
decision  making  is  implicated  in  most  of  these;  review  of  CC-130  accident  history, 
Reason’s  model*  of  organisational  issues  in  safety;  active  and  latent  failures;  course 
outline  -  focus  on  decision  making.  Introduce  the  PCT  and  IP  models.  Show  how  time 
pressure  determines  performance,  errors,  and  perceptions  of  workload.  Introduce 
decision  making  strategies,  and  the  speed  accuracy  trade-off.  Show  how  chronic 
physiological  and  psychological  stressors  can  effect  performance. 

Module  2:  Leadership  and  Followership 

Summary:  Teams  and  groups,  the  aviation  team,  leadership,  followership  (opportunities 
to  demonstrate  these  behaviours  will  occur  in  all  the  aspects  covered  in  this  training 
programme).  Authority  gradient  gives  control  priority  in  a  team  environment. 
Leadership  and  Followership  issues  will  be  addressed  in  each  module  as  appropriate. 

Module  3:  Superpilot,  dreamTeam  and  HercCrew 

Summary:  Introduce  SuperPilot  (an  all  capable  individual),  DreamTeam  (the  perfect 
team)  and  HercCrew  (the  ‘real  world’).  These  stereotypes  trace  the  emergence  of 
communication  and  Captaincy  as  one  goes  from  an  all  capable  individual,  to  teams  of 
people  from  different  backgrounds,  with  differing  levels  of  experience,  etc. 

MODULE  4:  Sensation/Perception 

Summary:  A  primer  on  how  the  world  is  perceived  and  how  perceptions  can  be  distorted 
by  illusions  or  expectations.  This  will  be  related  to  aviation,  and  examples  will  be  chosen 
to  represent  the  types  of  illusions  and  other  perceptual  distortions  that  aircrew  may 
encounter. 

Module  5:  Goal  Setting 


3  James  Reason’s  risk  management  model  is  discussed  in  some  detail  in  Volume  1  of  this  Report.  It  will  not  be 
elaborated  here. 
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Summary:  Goals  are  the  set  point  of  perceptual  control  loops  (compare  this  idea  with 
setting  room  temperature  on  a  thermostat);  they  are  how  the  human  wants  to  perceive 
(‘see’)  the  world.  Actions  are  driven  by  the  human’s  goals  and  shaped  by  their  state  of 
knowledge  or  mental  models. 

MODULE  6:  ACTION  SELECTION 

Summary:  Actions  are  the  manifestation  of  a  decision,  actions  are  intended  to  change  the 
state  of  the  world  so  that  perception  matches  goal;  this  is  usually  the  point  of  obvious 
departure  from  safe  operations  (unsafe  acts).  Actions  are  shaped  by  the  mental  model 
(compare  with  reaching  down  into  a  [mental  model]  toolbox  and  making  a  selection), 
make  the  distinction  between  knowing  and  doing  (confidence,  attitudes,  high  level  goals 
—  wishing  to  impress,  etc.  —  all  effect  which  tool  is  pulled  out  of  the  box).  Action 
selection  is  the  second  last  line  of  defence  in  risk  management.  Look  at  retrieving 
information  from  memory  and  what  can  go  wrong  (slips,  lapses  errors). 

Module  7:  Resource  management 

Summary:  An  overview  of  what  resource  is  being  managed  (attention,  time  and 
knowledge).  Review  of  SuperPilot,  DreamTeam  and  HercCrew  —  the  reason  for  multi¬ 
place  aircraft  is  that  the  task  load  is  too  much  for  one.  This  is  a  lead  in  to  the  next  three 
modules  which  deal  with  each  resource  to  be  managed  separately. 

Module  8:  Management  of  Control  and  Attention 

Summary:  Review  the  idea  of  a  hierarchical  system  with  control  switching  from  loop  to 
loop;  loops  not  attended  to  are  not  under  control;  there  is  a  need  to  sample  at  a  rate 
determined  by  how  quickly  things  can  change. 

Module  9:  Management  of  Time  pressure 

Summary:  Review  of  the  IP  model  and  the  effects  of  time  pressure  on  performance, 
workload,  and  errors. 

Module  10:  Management  of  Knowledge 

Summary:  Review  how  the  mental  model  shapes  actions,  and  that  decision  time  and  time 
pressure  depend  on  the  highest  level  of  individual  skills.  The  quality  of  a  decision 
depends  on  the  state  of  knowledge,  managing  knowledge  is  largely  managing  the  locus  of 
attention.  Having  access  to  all  relevant  available  knowledge  results  in  better  decisions. 
There  is  a  time  penalty  associated  with  gathering  this  knowledge. 

Module  11:  Communication 

Summary:  Review  the  functions  of  communication  (allocate  responsibilities  [AC], 
establish  common  goals  [AC],  build  common  mental  models  of  system  states  [All], 
provide  feedback  on  system  states  and  direct  attention  [All],  establish  and  maintain 
communication  channels  [All]).  Communication  is  more  than  the  words:  how  is  the 
information  coded;  draws  on  common  mental  models;  both  verbal  and  non-verbal 
communication  are  involved;  and  there  are  barriers  to  communication. 

A  You  are  here  button  is  shown  on  each  of  the  graphical  representations  of  the  IP  and  PCT 
models.  It  latches  each  module  to  the  theoretical  framework  and  repeatedly  returns  the  student  to 
the  underlying  structure  as  an  aid  to  comprehension  and  retention.  It  formally  makes  the 
connection  between  the  content  of  each  module  and  the  theoretical  framework.  It  justifies  the 
claim  that  each  module  derives  directly  from  the  underlying  theory. 
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FROM  KNOWLEDGE  TO  PRACTICE 


In  a  previous  Section,  a  theoretical  basis  for  HFDM  training  was  presented.  This  framework 
assembled  and  integrated  a  large  body  of  knowledge  from  the  human  factors  arena.  It  should  be 
possible  to  develop  skills  for  improving  safe  flight  performance,  by  the  appropriate  mix  of 
training  interventions,  from  this  framework.  This  assertion  can,  and  should,  be  tested 
empirically  by  some  form  of  assessment  system.  Strategies  for  assessing  the  effects  of  various 
human  factors  interventions  is  the  subject  of  Volume  2  of  this  report,  and  will  not  be  dealt  with 
in  detail  here.  However,  an  attempt  has  been  made  to  identify  observable  behaviours  associated 
with  the  core  skills  that  this  training  is  expected  to  build  (see  Appendix  1  of  this  Volume  for 
more  details).  The  presence  or  absence  of  these  behaviours,  or  the  degree  to  which  they  are 
present,  provides  a  basis  for  assessment. 


Knowledge  Development 

theory  —>  through  instruction/case  studies/examples  —>  knowledge 

The  first  stage  of  the  educational  process  is  the  building  of  a  body  of  knowledge  in  each  student 
from  which  skills  can  be  developed.  Traditionally  this  is  done  through  formal  classroom  training 
backed  up  with  directed  or  self-directed  study  of  books  or  manuals.  The  knowledge  built  at  this 
stage  should  facilitate  later  KRS-based  problem  solving  —  when  knowledge  has  been  developed 
into  skills  and  skills  are  being  exercised  in  an  operational  setting.  As  this  is  also  the  first 
exposure  to  this  material,  the  focus  should  be  on  presenting  core  concepts  that  can  be  expanded 
on,  mainly  in  terms  of  skill  development,  during  upgrade  and  continuation  training. 

To  support  this  phase,  detailed  course  notes  and  teaching  materials  are  required.  They  must 
present  the  core  material  in  a  concise  and  understandable  fashion.  This  is  a  challenge  for 
material  that  comes  from  the  psychological  and  human  performance  field  of  research  —  a  field 
that  might  be  treated  with  some  suspicion  by  crews  steeped  in  the  more  tangible  world  of  the 
physical  scientist  and  engineer.  As  the  IP  and  PCT  models  are  derived  from  engineering 
concepts,  it  is  expected  that  they  will  provide  a  bridge  between  these  two  worlds. 

The  knowledge  development  phase  should  cover  all  topics  presented  in  the  draft  syllabus.  The 
level  of  detail  covered  in  each  module  ideally  should  be  tailored  to  the  training  cycle,  making 
each  module  relevant  to  the  crewmember’s  current  stage  of  development  {just-in-time  training). 
With  traditional  CRM,  it  has  been  generally  accepted  that  all  crew  positions  should  be 
represented  during  training  exercises.  This  would  limit  the  approach  to  a  two  tiered  programme, 
that  is,  an  introductory  level  course  during  ab  initio  training  (in  conjunction  with  AC  upgrades) 
with  the  second  tier  coming  during  operational/conversion  training.  As  the  implementation  plans 
do  not  extend  outside  the  CC-130  community  at  the  present  time,  the  total  knowledge  base 
shown  in  the  draft  syllabus  must  be  presented  to  crews  at  their  first  exposure  under  such  a  two 
tiered  system.  Upgrade  and  recurrent  training  would  build  on  this  existing  knowledge  base 
(reviewing  and  elaborating  on  previously  presented  material  as  appropriate)  with  further  skill 
development,  case  studies  and  examples. 
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However,  if  the  knowledge  building  and  skill  development  phases  of  training  are  clearly 
separated,  a  more  reasonable  attempt  at  a  multi-tiered  and  tailored  system  can  be  made  (see 
Figure  3  as  an  example  of  a  multi-tiered  system ).  Figure  3  is  presented  for  discussion  only,  and 
does  not  represent  the  only  configuration  for  a  tailored  HFDM  training  programme. 


Pilot ,  NAV,  FE  and  LM  conversions 
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Squadron 


HFDM 

Classroom 

Training 


, CoPilots,  NAVs,  FEs,  LMs 
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ACs,  CoPilots,  X 
FEs¥  ▼ 
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Sessions 
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AC  Upgrade 
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Figure  3.  A  multi-tiered  approach  to  HFDM  training. 


In  the  approach  suggested  by  Figure  3,  the  detailed  core  knowledge  is  presented  once  only  as 
part  of  conversion  training  for  pilots,  navigators,  FEs,  and  LMs.  This  core  knowledge  is 


reviewed  and  elaborated  during  AC  upgrade  training,  and  biennial  crew  re-qualification  training. 
Little  skill  development  is  undertaken  during  the  classroom  sessions.  Skill  development  will  be 
left  to  hands-on  training. 


Skill  Development 


knowledge  —>  through  exercises/role  plays  — *  skills 

Skills  are  built  through  the  exercise  of  higher  level  knowledge.  In  PCT  terms,  processing  loops 
adapt  (learn)  only  when  control  is  exercised  and  attention  is  allocated.  Skills  allow  us  to  do 
things  as  distinct  from  just  knowing  about  things.  Skill  acquisition  is  demonstrated  by  the 
occurrence  of  target  behaviours.  Feedback  is  an  essential  part  of  this  process  because  feedback 
closes  the  loop  and  allows  the  adaptation  to  take  place.  The  word  skill  is  again  used  in  the 
colloquial  sense  rather  than  the  more  specific  sense  of  Rasmussen’s  KRS-hierarchy.  Skills  can 
be  developed  in  a  variety  of  hands-on  environments.  For  example: 

•  role  playing  and  team  exercises; 

•  analysis  of  case  studies; 

•  low  and  high  fidelity  simulation; 

•  instructional  rides;  or 

•  OJT. 

The  success  of  each  method  depends  on  the  particular  skill  being  developed.  Because  each  crew 
member  controls  slightly  different  decision  making  loops,  it  is  at  the  level  of  skill  development 
where  there  exists  the  greatest  opportunity  for  tailoring  training  to  the  individual.  Individual 
training,  by  position,  would  concentrate  on  those  areas  where  skill  sets  do  not  overlap.  For 
example: 


Position 


AC 


CP,  NAY,  FE,  LM 


Training  should  emphasise 

Strategic  DM. 

Risk  management. 

Application  of  rules  and  regulations. 

Appropriate  use  of  authority. 

Allocation  of  responsibilities  —  maximising  team 
performance. 

Establishing  and  achieving  common  goals. 

Team  building. 

Crew  expectations. 

Tactical  DM. 

Establishing  and  achieving  tactical  goals. 

Providing  supportive  technical,  interpersonal  and 
cognitive  support  to  other  team  members  and  the  AC. 

Contribution  to  the  team. 

Appropriate  assertiveness. 

AC  expectations. 
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This  will  require  additional  knowledge  building  for  ACs,  which  is  shown  in  Figure  3  under  the 
label  AC  Upgrade  Training.  While  the  knowledge  building  phase  is  shown  in  Figure  3  as 
occurring  in  isolation  (AC  upgrade  course  only)  from  other  team  members,  the  skill 
development/assessment  is  shown  as  occurring  in  a  team  environment  (role  playing,  exercises, 
case  studies).  It  is  during  the  doing  phase  that  it  is  essential  to  have  all  positions  present.  One 
can  not  build  team  skills  effectively  away  from  a  team  environment. 

In  the  article  titled  “CRM  and  the  Emperor’s  New  Clothes”,  John  Wise  attempts  to  make  the 
CRM  community  take  stock  and  collectively  ask  some  probing  questions  about  programme 
evaluation  [28].  Intended  to  be  controversial,  the  paper  argues  that  the  only  way  to  be  sure  that 
behaviour  has  indeed  changed,  is  by  direct  measure  of  the  behaviours  in  question.  This  is  the 
theme  of  many  contemporary  writers  on  CRM  and  in  fact  underlies  Helmreich’s  fifth  generation 
of  CRM  training  [6].  A  complementary  argument  has  been  pursued  in  the  situation  awareness 
literature  (e.g.,  see  Ref.  [29])  that  if  you  want  to  know  a  person’s  situation  awareness  or  state  of 
knowledge,  make  them  do  something  that  draws  on  that  knowledge  and  results  in  a  observable 
(and  therefore  testable)  response.  In  a  parallel  argument  it  is  possible  to  take  the  position  that  if 
you  want  to  test  to  see  if  a  skill  is  present,  then  invoke  situations  that  draw  on  that  skill  and  focus 
on  the  presence  or  absence  of  associated  behaviours.  In  the  draft  HFDM  syllabus  presented  in 
Appendix  1  to  this  Volume,  an  attempt  has  been  made  to  identify  a  subset  of  potentially 
observable  behaviours  for  each  module.  The  presence  or  absence  of  these  behaviours  could  be 
used  to  evaluate  the  acquisition  of  skill.  A  more  global  approach  to  assessment  is  described  in  a 
companion  Volume  (see  Volume  2  to  this  report). 


ACHIEVING  SAFE  FLIGHT  PERFORMANCE 


skills  —>  through  organisational  reinforcement  —>  everyday  practice 

The  assumption  of  any  aircrew  training  system  is  that  safe  and  effective  flight  performance  will 
be  achieved  when  crews  reliably  and  regularly  use  the  target  behaviours  instilled  by  the  training. 
Given  that  the  human  has  a  large  toolkit  of  behaviours  to  chose  from  on  any  given  occasion, 
some  form  of  organisational  re-enforcement  is  needed  to  ensure  that  target  behaviours  are 
consistently  used  in  preference  to  non-target  or  even  undesirable  behaviours.  Organisations  can 
provide  re-enforcement  in  a  variety  of  ways,  for  example,  by  calling  for  general  flying  stand- 
downs,  grounding  specific  fleets,  or  by  mounting  sporadic  safety  seminars  and  programmes. 
Such  interventions  are  often  initiated  by  a  run  of  incidents  or  accidents  in  a  condensed  period  of 
time-  (e.g.,  the  USAF  called  a  brief  flying  stand-down  after  a  series  of  major  losses  during  a  one 
week  period  in  September  1997). 

While  this  type  of  intervention  may  have  the  desired  effect  in  the  short  term,  it  is  likely  that  its 
benefits  will  be  short-lived.  To  have  long-term  effects,  there  must  be  a  permanent  change  in  the 
organisation  and  safety  related  behaviours  of  those  within  the  culture.  More  enduring 
organisational  interventions  include. 

•  The  establishment  of  rules  and  regulations  (sometimes  only  as  good  as  the  last  accident). 

•  A  clearly  stated  set  of  organisational  values  (these  establish  high  level  goals  and  therefore 
shape  and  direct  behaviour),  for  example: 

—  operational  requirements  are  not  to  take  precedence  over  safe  operations  during 
peacetime; 

—  building  technical  excellence  and  professionalism  are  goals  of  the  organisation. 
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•  An  organisation  that  is  seen  to  act  in  accordance  with  their  stated  values,  for  example  by: 

—  providing  resources  for  training  and  technical  development; 

—  rewarding  those  that  display  excellence  and  professionalism  and  not  rewarding  those  who 
do  not; 

•  Establishment  of,  and  assessment  against,  standards. 

•  Monitoring  how  the  organisation  is  performing  against  those  standards  and  the  provision  of 
regular  howgozit  feedback. 


As  one  of  the  organisational  support  mechanisms  listed  above  is  based  on  the  commitment  to  the 
training  system,  an  attempt  was  made  to  estimate  the  resources  required  to  implement  the  type  of 
programme  outlined  in  Figure  3.  The  following  contains  the  suggested  duration  and  frequency 
for  each  stage  of  training. 


Stage 

Requirements 

(Frequency) 

Objectives 

HFDM  Classroom  Training 

3  days  (on  initial 
conversion) 

Develop  an  understanding  of  core 
knowledge  on  human  decision  making 
and  information  processing. 

Role  Playing,  Exercises  and 
Case  Studies 

2  days  (on  initial 
conversion,  then  on 
biennial  re¬ 
qualification,  and 

AC  upgrade) 

Develop  high  level  skills  in  situation 
assessment  and  analysis.  This  stage 
should  immediately  follow  the  classroom 
training.  For  the  first  day,  apply  core 
knowledge  to  case  studies.  Each  team 
member  to  analyse  the  situation  from  the 
perspective  of  their  own  crew  position  if 
represented.  If  their  position  is  not 
represented  they  should  pick  one  that  is. 
On  the  second  day  exercises  will  focus  on 
team  DM  in  time  pressured,  uncertain 
environments. 

Simulator  Sessions 

2  LOS  /  LOE  type 
scenarios  (initial 
conversion, 
biennial  re¬ 
qualification,  and 

AC  upgrade) 

ACs,  CPs  and  FEs  to  run  through  at  least 

2  LOS/LOE  type  simulations,  with  full 
audio/video  debrief.  These  scenarios 
should  emphasise  DM  and  the  need  to 
process  information  at  each  of  the 
awareness,  consider  the  implications  and 
make  a  plan  stages  of  the  Awareness 
Implications  Plan  (AIP)  decision  model. 

OJT 

Ongoing 

During  OJT  ACs  should  take 
opportunities  to  vocalise  their  decisions  in 
AIP  form  and  test  other  crewmembers 
against  this  template. 

Check  Rides 

As  required 

Check  rides  should  test  DM  skills  as  well 
as  technical  skills. 
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Refresher  Training 

2  days  (annually) 

Refresher  training  should  review  the 
HFDM  theoretical  framework  and  expand 
one  or  more  aspects  that  have  relevance  to 
the  Squadron’s  operations  (e.g.,  perhaps 
there  have  been  some  incidences  where 
visual  illusions  have  played  a  part,  or  a 
number  of  decisions  have  been  made  that 
showed  the  confirmation  bias  at  work). 
Case  studies  and  exercises  will  be  used  to 
refresh  situation  assessment  skills. 

Re-qualification  Training 

1  day§  (biennially) 

Re-qualification  training  should  have 
similar  objectives  to  refresher  training. 

The  topics  for  deeper  study  may  be 
chosen  by  the  training  squadron  on  the 
basis  of  deficiencies  they  have  seen  in 
simulator  and  classroom  settings. 

AC  Upgrade  Training 

1  day§  (in 
conjunction  with 

AC  upgrade  course) 

This  will  review  the  theoretical 
framework  and  expand  on  topics  related 
to  strategic  DM,  risk  management, 
allocation  of  responsibilities,  etc. 

Notes: 

§  Followed  by  role  playing,  exercises,  case  study  analysis  in  a  team  environment 
and  LOS/LOE  scenarios  in  the  simulator  with  detailed  debriefing  playing  the  key 
role. 

Figure  3  represents  one  of  at  least  several  ways  of  dividing  up  the  HFDM  training  budget, 
therefore  the  preceding  outline  is  for  discussion  purposes  only.  A  programme  such  as  the  one 
outlined  in  Figure  3  would  make  the  following  additions  to  current  CC-130  training 
requirements: 

•  1  extra  classroom  day  added  to  the  current  ACT  training  course; 

•  2  extra  days  devoted  to  hands-on  team  DM  training  for  all  positions  (as  preparation  for 
simulator  sessions  in  the  case  of  the  ACs,  CPs  and  FEs); 

•  2  days  per  year  of  HFDM  refresher  training  within  the  Squadrons  (this  could  be  in 
conjunction  with  a  safety  stand-down);  and 

•  3  extra  days  for  all  those  going  through  AC  upgrade  training  or  re-qualification  training. 

Manning  levels  within  the  training  Squadron  to  accomplish  this  task  have  not  been  assessed. 
Overall  these  are  modest  additions  to  a  training  programme  that  is  currently  devoted  almost 
entirely  to  technical  issues.  Considering  that  human  factors  (HF)  failures  are  assigned  to 
somewhere  between  70%  and  80%  of  the  accidents  and  incidents  around  the  world,  the  proposed 
amount  of  time  spent  in  HF  training  is  not  excessive. 


A  BASIS  FOR  ASSESSMENT 

While  the  issue  of  programme  assessment  is  addressed  in  more  detail  in  Volume  2  of  this  report, 
the  implications  of  the  theoretical  framework  offered  by  the  IP/PCT  model,  to  the  development 
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of  assessment  instruments,  is  worthy  of  comment.  The  causal  chain  from  the  IP/PCT  model  is 
argued  to  be  as  follows. 

•  Decisions  are  the  end  point.  Their  timeliness  and  appropriateness  sets  the  gold  standard 
against  which  crew  system  performance  is  always  judged. 

•  If  an  individual  has  an  appropriate  goal  and  the  necessary  base  knowledge,  given  time  to 
process  all  the  available  information,  an  appropriate  decision  is  expected. 

•  Once  the  conditions  for  goals  and  knowledge  are  established,  it  is  the  timeliness  of  the 
information  processing  that  is  at  issue.  Timeliness  (through  its  effect  on  time  pressure)  of 
decision  making  becomes  the  driver  of  performance,  errors,  and  perceived  workload  when 
externally  paced. 

•  Timeliness  depends  on  the  state  of  the  pre-existing  mental  model  (level  of  proficiency/skill) 
and  the  management  of  attention  (situation  assessment). 

Failures  to  make  timely  and  appropriate  decisions  under  high  time  pressure  can  be  traced  to: 
inappropriate  goal  setting;  the  lack  of  pre-requisite  or  base  knowledge  (forcing  the  use  of  R  and 
K  instead  of  S  processing);  or  poor  situation  assessment  due  to  a  restricted  locus  of  control 
arising  from  the  time  pressure.  Failures  under  low  time  pressure  can  usually  be  traced  to 
deficiencies  in  the  locus  of  control/attention  due  to  complacency  or  vigilance  effects. 

A  framework  for  a  measurement  system  might  look  something  like  the  following  (the  assessment 
criteria  listed  below  are  a  subset  of  the  skills  presented  in  Appendix  1). 

Type  of  measure  functional  Area  Assessment  criteria 

Outcome  Decision  Making  Individual 

•  Were  the  decisions/actions  appropriate? 

•  Did  the  decisions/actions  occur  within 
the  time  available? 

Process  Time  Management  Individual 

•  Did  each  crewmember  appear  calm  or 
rushed? 

•  Did  each  crewmember  prioritise, 
delegate,  delay,  shed,  buy  time 
appropriately? 

* Is  each  crewmember  using  the 
appropriate  level  of  KRS  processing  for 
the  task  in  hand? 

•  Did  each  crewmember  act  to  relieve  the 
loads  of  other  crewmembers. 

Crew 

•  Was  a  balance  achieved  in  the  imposed 
time  pressure  of  all  crewmembers? 
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Knowledge  Management 


Attention  Management 


Team  Process 


•  Did  each  crewmember  actively  seek  and 
correctly  perceive  (was  Aware  of)  all 
important  information? 

•  Did  each  crewmember  consider  the 
Implications  of  the  situation? 

•  Did  each  crewmember  form  Plans  to 
cope  with  the  situation  and  its 
implications,  appropriate  to  their  role? 

•  Did  each  crewmember  set  appropriate 
goals  and  communicate  them  as 
required  (e.g.,  by  briefing)? 

•  Did  each  crewmember  share  their 
knowledge  (i.e.,  establish  a  common 
mental  model)  as  appropriate? 

•  Did  each  crewmember  resolve  conflicts 
appropriately? 

•  Did  each  crewmember  consider  all  the 
information  they  were  aware  of  in 
making  their  decisions? 

Crew 

•  Were  the  crew  working  from  common 
goals  and  mental  models? 


•  Did  each  crewmember  attend  to 


(control)  the  variables  within  their 
domain  of  responsibility? 

•  Did  each  crewmember  direct  attention 


when  appropriate? 


Crew 


•  Did  the  crew  ensure  that  goals  were 
achieved  by  monitoring, 
acknowledging,  cross-checking, 
backing  up,  etc? 


•  Did  each  crewmember  provide 
appropriate  supportive  technical, 
interpersonal  and  cognitive  skills? 

Crew 

•  Was  the  crew  cohesive? 


•  Was  an  environment  established  that 


was  open,  receptive,  supportive,  non¬ 
threatening? 


A  variety  of  assessment  instruments  can  be  developed  from  this  framework.  Behaviourally 
Anchored  Rating  Scales  (BARS)  have  a  history  in  CRM  evaluation  [23].  BARS  contain  written 
descriptions  of  the  behaviours  associated  with  each  of  the  rating  scale  values.  This  is  said  to  aid 
evaluators  in  applying  the  scales,  and  to  result  in  improved  inter-  and  intra-rater  reliability.  The 
US  Army  has  used  BARS  extensively  in  the  evaluation  of  their  ACT  training  programme,  and 
the  USAF  is  planning  to  use  BARS  in  a  large  scale  research  programme  to  evaluate  MC-130P 
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team  performance  in  combat  mission  training  [23].  BARS  can  evaluate  on  multiple  levels  (say 
1-5)  or  on  a  simple  YES/NO  basis. 

A  series  of  BARS  could  be  developed  from  the  framework  presented  above,  for  each  crew 
position  (AC,  CP,  FE,  etc.),  to  be  administered  on  a  segment  by  segment  basis  or  for  the  mission 
as  a  whole.  These  might  range  from  a  simple  5  factor  instrument  (Decision  Making,  Time 
Management,  Knowledge  Management,  Attention  Management,  Team  Process)  to  a  detailed  21 
factor  instrument  that  rates  each  criterion  behaviour  separately.  The  5  factor  instrument  would 
be  appropriate  for  operational  people  (trainers,  check  pilots,  standards  officers,  etc.)  while  the  21 
factor  instrument  would  most  likely  be  used  by  researchers.  When  done  on  a  segment  by 
segment  basis,  the  specific  tasks,  required  knowledge,  implications,  plans  and  locus  of  control 
could  be  listed  for  that  segment  thus  adding  to  the  potential  diagnosticity  of  the  instrument. 

In  Appendix  3,  an  example  5  factor  BARS  is  presented  to  demonstrate  the  concept.  This 
instrument  has  not  been  validated  and  is  presented  merely  to  show  how  the  complexity  of  the 
IP/PCT  model  can  be  distilled  down  to  an  instrument  for  use  in  real-time  operational  evaluation. 

The  development  of  such  an  instrument  completes  the  sequence  from  theory  — >  course  syllabus 

— >  evaluation,  all  within  the  IP/PCT  paradigm.  The  behavioural  anchors  that  are  used  in  this 
example  are  notional  at  this  point,  and  should  have  operator  input  to  make  them  more  relevant  to 
the  CC-130  community’s  needs  for  standards  and  evaluation.  Critical  to  this  process  will  be  the 
development  of  definitions  for  the  minimum  required  skill  levels. 

It  should  be  noted  that  the  dimensions  of  this  scale  are  conceptually  independent.  For  example: 

•  the  appropriateness  of  a  decision  can  be  judged  independently  of  the  timeliness  in  most  cases 
(one  can  have  an  appropriate  decision  that  is  either  timely  or  late,  etc.); 

•  outcome  and  process  can  be  separated  (a  timely  and  appropriate  decision  might  result  from 
either  good  situation  assessment  —  good  locus  of  control,  etc.  —  or  from  serendipitously 
attending  to  a  critical  piece  of  information);  and  finally 

•  time  management,  knowledge  management  and  attention  are  separate  entities. 

In  practice,  and  over  many  observations,  ratings  on  the  scales  are  likely  to  be  correlated  as  time, 
knowledge  and  attention  management  trade-off,  one  against  the  other.  However,  the  notional 
independence  of  the  underlying  concepts  should  make  the  scale  easier  to  use  and  be  more 
powerful  from  a  diagnostic  point  of  view. 

In  contrast  to  the  categories  arising  from  the  IP/PCT  model  (see  above),  the  USAF  proposed  the 
following  five  functional  areas  (dimensions)  for  measuring  CRM  performance  in  their  MC-130P 
operations  [23]. 

1.  Time  Management  (TM):  Involves  the  ability  of  the  combat  mission  team  to  employ  and 
manage  limited  time  resources,  so  that  all  tasks  receive  sufficient  time  to  be  performed 
correctly  and  critical  tasks  are  not  omitted. 

2.  Tactics  Employment  (TE):  Includes  all  analytic  activities  necessary  to  avoid  or  minimise 
threat  detection  or  exposure,  and  to  successfully  co-ordinate  complex  mission  events  and 
multiple  mission  objectives. 

3.  Function  allocation  (FA):  Includes  the  division  of  crew  responsibilities  so  that  workload  is 
distributed  among  the  crew,  avoiding  redundant  tasking,  task  overload,  and  crewmember 
disinterest  or  non-involvement.  Tasks  should  be  allocated  in  such  a  manner  so  that 
crewmembers  are  able  to  share  information  and  co-ordinate  activities. 

4.  Situation  Awareness  (SA):  Entails  maintaining  an  accurate  mental  picture  of  mission  events 
and  objectives  as  they  unfold  over  time  and  space.  Emphasis  and  analysis  are  placed  on  the 
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three  levels  of  SA  (perception,  integration,  and  generation)  and  their  impact  on  team  co¬ 
ordination. 

5.  Command,  Control  and  Communication  (C3):  Encompasses  those  activities  required  to 
involve  external  parties  in  the  mission,  and  to  maintain  communications  with  those  external 
team  members,  communication  within  the  crew,  and  controlling  the  sequence  of  mission 
events  according  to  the  mission  execution  plan. 

Here  the  domains  seem  to  be  inherently  related.  TM,  FA,  and  the  command  and  control  parts  of 
C3,  all  have  concepts  of  timeline  management  associated  with  them.  FA  and  SA  also  overlap  in 
the  area  of  mental  model  building,  particularly  the  shared  mental  model.  TE  and  C3  both  have 
command  or  co-ordination  aspects  built  into  them.  There  appears  to  be  a  mixture  of 
measurements  at  what  might  be  conceived  to  be  both  the  crewness  and  the  individual  level, 
within  and  across  scales.  Yet  there  is  no  particular  structure  to  aid  in  diagnosis.  This  would 
make  it  difficult  to  debrief  the  individuals  that  make  up  the  crew  —  note  that  we  can  only  change 
crew  performance  by  changing  the  behaviours  of  those  individuals  within  the  crew. 
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CONCLUSIONS 


This  Volume  describes  a  proposed  syllabus  for  a  course  in  Human  Factors  in  Decision  Making. 
The  proposed  course  attempts  to  address  some  of  the  criticisms  levelled  at  early  generation  CRM 
programmes  —  such  as  their  emphasis  on  process  rather  than  observable  product.  Its  scope  is  on 
one  hand  broader  than  traditional  CRM  training,  but  on  the  other  hand  more  focused  by  the 
specific  emphasis  on  decision  making.  The  draft  syllabus  is  made  up  of  1 1  modules  covering  a 
variety  of  topics,  each  one  critical  to  the  understanding  of  human  decision  making.  These 
modules  are  titled:  an  introduction  to  the  decision  making  loop;  leadership  and  followership;  the 
emergence  of  communication  and  captaincy  in  teams  and  groups;  sensation/perception;  goal 
setting;  action  selection;  management  of  control,  attention,  time,  and  knowledge;  and  finally 
human-human  and  human-machine  communication. 

The  distinction  between  the  proposed  training  and  other  similar  endeavours  lies  in  its  foundation 
in  theory.  Two  strong  information  processing-based  models  (the  Information  Processing  [IP] 
model  and  the  Perceptual  Control  Theory  [PCT]  model)  form  the  theoretical  framework  for  the 
proposed  programme.  This  provides  a  consistency  that  permeates  through  all  course  modules 
and  is  expected  to  aid  understanding  and  retention.  Both  the  course  syllabus  and  the  potential  for 
developing  assessment  instruments  derive  directly  from  the  IP/PCT  paradigm. 
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APPENDIX  1 :  DRAFT  SYLLABUS  FOR  HUMAN  FACTORS  IN  DECISION 

MAKING  TRAINING 
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Introduction  to  the  Decision  Making  Loop 

Introduction:  accidents  statistics,  majority  of  accidents  attributed  to  human  factors 
issues,  decision  malang  implicated  in  most  of  these,  review  of  CC-130  accident  history 
Reason  s  model,  active  and  latent  failures,  course  outline  —  focus  on  decision  making. 


Communication 


Captaincy 
Rules  and  regulations 
Resource  mangement 
Crew  Coordination  etc. 


This  Course 


Decision 

Making 


The  PCT  Model  and  the  Decision  Making  Loop:  introduction  to  the  thermostat  and 
clas^d  loop  control  of  temperature  (temperature  set  point,  etc.)  leading  to  the  PCT  model, 
goals,  mental  models,  situation  awareness  and  the  shaping  of  decisions,  essential 
requirement  for  feedback  in  goal  directed  activity.  Relate  the  components  of  the  air- 
conditioning  control  system  (something  that  is  familiar)  to  the  components  of  the 
perceptual  control  loop  (a  new  concept). 


Air  Conditioning  System 

•  Thermostat  set  point 

•  Temperature  and  humidity  sensors 

•  Valves,  compressor,  heat  exchanger 
properties 

•  Rated  compressor  power  (BTU) 

•  Physical  limits  on  cooling  capacity 
(size  of  compressor,  heat  exchanger, 
wear,  etc.) 


Human  Decision  Maker 

•  Goals 

•  Eyes,  ears,  sense  of  smell  and  touch 

•  Mental  models  of  action  selection 

•  Internal  processing  rate  (bits  per 
second) 

•  Various  physiological  and 
psychological  stressors  (fatigue, 
motivation,  anxiety,  etc.) 
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The  IP  Model:  introduction  to  the  IP  model,  effects  of  time  pressure  on  performance  and 
errors,  strategies  and  the  speed  accuracy  trade  off,  errors  of  omission,  skills  rules  and 
knowledge  based  behaviours,  recognition  primed  decision  making.  Introduce  the 
integration  of  PCT  and  the  IP  model,  time  pressure  and  the  bandwidth  of  the  loop, 
knowing  the  goals  and  mental  model,  behaviour  is  predictable. 


Introduction 


4»a*'  4**=*;  4*8*'  4ss;i;  <a*m  <asr  <aii  4am  4iir  4s  m 


Divided  by 


Equals  23 


[jEElSEiSiES  Divided  by  [jJ5£|2Z2EE3  Equals 


'-Urn®  Pressure 


Adaptation  to  Excessive  Time  Pressure 


4»b«:  <a»r  43:1:1;  4b  sit;  41111  4b;s;ii;:  4m  4am  <a*i  4si:»  <2«»: 


decreasing 


Divided  by 


Task  Difficulty 


adapts  Effort 


increasing 


aSBBSlgSB  Divided  by  |jJ2]^1222222Si Ecluais 


TimePressure 


45 


Time  Pressure  and  Errors 


Time  Pressure  and  the  Speed/Accuracy  Trade  Off 
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Effects  of  Physiological  and  Psychological  Stressors 
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Teams:  multi-person  PCT,  requirements  for  stability,  need  for  compatible  goals  and 
mental  models,  authority  gradients,  emergence  of  role  and  task  allocation. 

Introduction 


Multiple  Controllers  (Human  and  Machines) 


Living  with  Error:  living  with  error  (‘error’  is  inevitable),  breaking  the  chain, 
monitoring/feedback,  closed  loop  behaviour  allows  us  to  be  successful  without  perfect 
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mental  models,  open  loop  behaviours  lack  error  checking,  you  as  the  aircrew  are  the  last 
line  of  defence  in  the  accident  chain,  managing  error. 

Summary: 

Knowledge:  Basic  understanding  of 

1.  effects  of  time  pressure  on  workload,  performance,  and  errors, 

2.  strategies  for  managing  time  pressure, 

3.  the  concept  of  closed  loop  control  and  the  importance  of  feedback, 

4.  requirements  for  stable  team  performance;  and 

5.  the  concept  of  goals  and  mental  models. 

Skills:  NA. 

Assessment: 

1.  core  knowledge. 

Resources: 


Exercises 
Case  Studies 

Videos 
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leadership  and  followership 

Introduction:  teams  and  groups,  the  aviation  team,  leadership,  followership 
(opportunities  to  demonstrate  these  behaviours  will  occur  in  all  the  aspects  covered  in 
this  course),  authority  gradient  gives  control  priority  in  a  team  environment. 
Responsibility:  legal  responsibility  for  the  safety  of  the  operation  vested  in  AC, 
responsibility  of  the  AC  and  the  crew. 

Authority:  origins  of  power,  authority  gradients  (flat,  negative  and  positive)  and  effects 
on  the  team. 

Leadership:  ability  to  influence  others,  differences  between  leadership  and  authority, 
leadership  styles,  situational  leadership,  maturity  models. 

Followership:  followership  doesn’t  imply  subservience,  responsibility  to  support  AC’s 
decision  as  long  as  safety  is  not  jeopardised,  when  to  dispute  non-safety  related  decisions, 
proactive  followership,  can  all  lead? 

Dimensions  of  Captaincy:  From  the  Captaincy  study. 

Summary: 

Knowledge:  Understanding  of 

1 .  the  Captain’ s  role  and  responsibility 

2.  the  crew’s  role  and  responsibility. 

Skills: 

1.  Leadership. 

2.  Followership. 

Assessment: 

1.  Core  knowledge. 

2.  Acts  in  accordance  with  responsibilities. 

3.  Appropriate  exercise  of  authority. 

4.  Demonstration  of  Captaincy  behaviours. 

5.  Ability  to  resolve  conflict. 

6.  Proactive  support  of  AC’s  decisions  through  technical,  interpersonal  and  cognitive 
skills. 

7.  Appropriate  challenge —  demonstrating  when  to  challenge  and  when  to  demure. 
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Resources: 


Exercises 


Case  Studies 


Videos 


1.  Discuss  leadership  and  followership. 

2.  Have  each  class  do  a  mini-version  of  generating 
Captaincy  dimensions  for  themselves  (as  individuals  or 
as  groups  —  clearly  should  be  done  prior  to  a 
presentation  of  results  —  and  see  how  they  match  up 

Core  Knowledge 

1 .  The  rep  grid  study  of  Captaincy. 

2.  CC- 130  study  results. 

Positive 

1.  BA  Indonesia 

2.  Horizon  2658,  Seattle,  WA 

3.  United  811,  Honolulu,  HI 
Negative 

1.  HTI  CC- 130  example. 

2.  Jackson’s  Hole  CC-130. 

3.  AA  965,  Cali,  Columbia. 

4.  Tiger  66,  Kuala  Lampur,  Malaysia. 

5.  FAA,  Front  Royal,  VA. 

Positive 

1 .  CC- 130  example  of  AC  dealing  with  strategic  DM  and 
crew  backing  up  with  pro-active  tactical  decision 
making. 

Negative 

1 .  Northwest  1451,  Detroit, 
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SuperPilot,  Dream  Team  and  HercCrew 

Introduction:  introduce  SuperPilot,  DreamTeam  and  HercCrew. 

Emergence  of  CRM  and  Communication:  trace  the  emergence  of  CRM  and 
communication  as  one  progresses  from  SP  to  DT  to  HercCrew,  functions  of 
communication,  information  and  uncertainty. 

Summary: 

Knowledge:  An  understanding  of  the  following  concepts 

1.  Functions  of  Captaincy. 

2.  Functions  of  communication. 

Skills:  NA. 

Assessment: 

1.  Core  knowledge. 
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Sensation/Perception 


Introduction,  perception  is  the  result  of  passing  the  incoming  sensory  information  (light, 
sound,  force,  texture,  etc.)  through  a  transformation  process  (some  hard  wired,  some 
learned  at  an  early  age,  some  acquired  over  time),  how  you  perceive  the  world  may  not 
be  how  it  is  (illusions),  the  same  stimulus  can  elicit  different  sensations  depending  on  set 
expectation,  etc. 

Miss-perceptions:  when  the  perception  may  not  match  the  ‘reality’,  visual  illusions, 
errors  in  sound  localisation,  aviation  related  illusions  and  effects  on  control,  spatial 

somogravic  effects,  miss-perception  of  auditory  information,  effects 
of  displays  (moving  aircraft  versus  moving  horizon  AI,  moving  scale  versus  moving 
pointer  and  control  reversals,  radar  reversal,  etc.). 

Leadership/Followership  Issues:  be  aware  of  the  potential  for  misleading  sensations, 
brief/warn  cross  check  as  appropriate  if  someone  is  not  reacting  to  your  perception  of  the 
world  (either  they  or  you  may  be  operating  under  the  wrong  mental  model  —  wrong 
model  =  wrong  action). 

Summary: 

Knowledge:  An  understanding  of 

1.  Common  aviation  related  miss-perceptions. 

2.  Review  of  disorientation  effects. 

Skills: 

1.  Ability  to  recognise  the  circumstances  that  may  result  in  ambiguous  perceptions. 

2.  Ability  to  maintain  control  under  conflicting  somogravic  stimuli. 

Assessment: 

1.  Core  knowledge. 

2.  Demonstrates  an  awareness  of  the  potential  for  ambiguous  sensory  cues,  by  warning 
briefing  or  checking. 

3.  Responds  to  the  actions  of  other  crew  members  when  seem  inappropriate  to  the 
situation  (may  be  indicative  of  a  miss-perception). 
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Resources: 


Exercises 


Case  Studies 


Videos 


1.  Generate  a  list  of  potential  situations  that  may  lead  to 
miss-perceptions. 

2.  Use  slides  to  show  effects  of  runway  widths,  slopes  on 
perceived  approach  angle. 

Positive 
1.  tba 
Negative 

1 .  W ainwright  AK. 

2.  Box  Top  22,  Alert,  NWT 

3.  The  tall  fireman. 

4.  US  Air  105,  Kansas  City,  MO 

1.  tba 
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Goal  Setting 


•  • '  ---I 


Difference 


Goals  -  What 
are  you  trying 
to  achieve 


How  you  think 
the  world  is 


Mental  Model  - 
output 


Mental 

Rehearsal 


Mental  Model  - 
input 


Introduction:  review  —  goals  are  the  set  point  of  your  perceptual  control  loops 
(compare  with  setting  room  temperature  on  a  thermostat),  they  are  how  you  want  to 
perceive  (‘see’)  the  world,  your  actions  are  driven  by  your  goals  and  shaped  by  your 
mental  model. 

Goals:  tactical  and  strategic  goals,  need  for  appropriate  goals,  goals  should  match  the 
capability  of  the  crew,  your  goals  are  your  first  line  of  defence  in  risk  management,  need 
to  communicate  and  gain  agreement  on  goals  (stable  multi-loop  control),  monitoring 
(loop  closure)  requires  an  awareness  of  goals. 

Rules  and  Regulations:  ask  the  question  “...is  this  legal?”  (push  down  below  MDA,  take 
off  in  low  RVR,  etc.). 

Leadership/Followership  Issues:  authority  and  responsibility,  who  sets,  AC’s  role  - 
crew’s  role,  maturity  model  of  leadership  revisited  in  this  context. 

Summary: 

Knowledge: 

1.  Concept  of  strategic  and  tactical  goals  and  how  they  relate  to  risk  management. 

2.  Roles  and  responsibilities  of  all  positions. 

3.  Knows  the  rales  that  are  appropriate  to  their  position. 

Skills: 

1.  Ability  to  set  and  communicate  appropriate  goals. 

Assessment: 

1.  Core  knowledge. 

2.  Demonstrates  appropriate  goal  setting  (strategic  and  tactical). 

3.  Communicates  and  gains  acceptance  of  goals. 

4.  Acts  in  accordance  with  the  rales  and  regulations  that  govern  the  operation. 


Resources: 


Exercises 


Case  Studies 


Videos 


1.  Have  all  participants  list  some  of  the  tactical  and 
strategic  goals  they  might  set  in  an  operational  setting  - 
discuss. 

2.  Groups  should  analyse  case  studies,  or  additional 
scenarios,  in  terms  of  (1)  initial  goal  setting  and  (2) 
changing  goals  as  the  scenario  unfolds  -  if  goals 
change  or  start  to  conflict  this  exercise  introduces  the 
notions  of  task  prioritisation/offloading/workload 
management,  etc. 

Positive 

1.  tba 

Negative 

1 .  Jackson’ s  Hole  CC- 1 30  accident. 

2.  Box  Top  22,  Alert,  NWT. 

3.  Bud  Holland. 

4.  Avianca  052,  New  York,  NY. 

5.  Eastern  L-101 1,  Miami,  FL. 

Positive 

1.  CC-130  crew  showing  changing  goals  as  the  situation 
changes. 

Negative 

1.  tba 
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Action  Selection 


Introduction:  the  task  is  not  done  until  the  goal  is  ‘seen’  to  be  achieved.  This  means  we 
must  act  on  the  world. 

Action  Selection:  manifestation  of  a  decision,  your  action  is  intended  to  change  the  state 
of  the  world  so  that  your  perception  matches  your  goal,  this  is  usually  the  point  of 
obvious  departure  from  safe  operations,  actions  are  shaped  by  the  mental  model  (this  is 
like  reaching  down  into  the  mental  model  toolbox  and  making  a  selection),  difference 
between  knowing  and  doing  (your  confidence,  attitudes,  high  level  goals  —  wishing  to 
impress,  peer  pressure,  etc.  —  all  effect  which  tool  you  pull  out  of  the  box),  second  last 
line  of  defence  in  risk  management,  retrieving  information  from  memory,  what  can  go 
wrong. 

Memory:  types  and  limitations  of  memory,  associative  model  of  memory,  slips,  errors  of 
commission,  memory  aids  (check  lists,  etc.),  need  to  confirm  and  resolve  discrepancies, 
spatial  and  verbal  encoders. 

Risk  Management:  every  decision  involves  risk  management,  risk  management  is  based 
on  goals,  the  state  of  knowledge  and  further  shaped  by  attitudinal  factors,  safe  and  unsafe 
attitudes  (over  confidence,  complacency,  get-home-itis,  macho,  etc.),  attitudes  and 
changing  attitudes,  knowledge  versus  experience,  compensation  for  lack  of  experience, 
many  tactical  goals  and  mental  models  may  achieve  the  same  strategic  goals,  rules  and 
regulations,  risk  management  is  about  bounding  behaviour  -  works  when  regulations 
don’t  coyer  the  situation,  danger  signs  (departure  from  SOPs,  etc.),  use  of  the  most 
conservative  response,  use  of  the  veto  for  ACs,  effects  on  your  decision  others  —  the  big 
picture  (e.g.,  in  formation). 

AIP  Model:  before  you  select  an  action  have  you  gone  through  the  process  of 
Awareness-Implications-Plan? 

Goal  Achievement:  timeliness  and  appropriateness  of  the  outcome  relative  to  your  goal. 
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Feedback:  must  close  loops,  essential  to  ensure  goal  achievement,  feedback  allows  the 
use  of  less  than  perfect  mental  models  (try  controlling  the  aircraft  with  your  eyes  shut), 
last  line  of  defence  in  error  management. 

Leadership/Followership  Issues:  responsibilities  for  risk  management,  all  have  the 
responsibility  to  monitor  the  situation  and  see  that  collectively  you  are  moving  towards 
the  goal  (particularly  with  respect  to  safe  operations). 

Summary: 

Knowledge: 

1.  Types  of  memory  lapses  and  aids  to  recall. 

2.  Signs  that  risk  management  is  breaking  down. 

3.  The  requirement  for  feedback  and  monitoring. 

4.  The  AIP  model. 

Skills: 

1.  The  ability  to  detect  the  symptoms  of  risk  management  breakdown. 

2.  Ability  to  use  the  AIP  model  in  a  number  of  exercises. 

Assessment: 

1.  Core  knowledge. 

2.  Acts  when  risk  management  is  breaking  down,  by  calling  attention  to  the  situation 
and  offering  solutions. 

3.  Demonstrates  the  use  of  the  AIP  model  in  decision  making. 
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Resources: 

Exercises 

Case  Studies 


Videos 


1 .  Use  AIP  in  a  scenario. 

2.  Use  of  feedback. 

Positive 

1 .  Pan  Am  543,  Boston 

2.  United  232,  Sioux  City,  IA 

3.  Aloha  1712  Maui,  Hawaii 
Negative 

1.  CC- 130  mid  air. 

2.  British  Midlands,  Manchester,  UK 

3.  Air  Inter,  Strasbourg,  France. 

4.  Delta  1141,  Dallas  Fort  Worth,  TX 

5.  Air  Bus  A3 10  Vancouver,  BC. 

6.  Avianca  052,  New  York,  NY. 

7.  Eastern  L-101 1,  Miami,  FL. 

Positive 

1.  CC-130  crew  using  an  AIP  approach  during  system 
failures. 

Negative 
1.  tba 
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resource  management 

Introduction:  review  of  SuperPilot,  DreamTeam  and  HercCrew,  reason  for  multi-place 
aircraft  is  that  the  task  load  is  too  much  for  one.  This  module  introduces  the  3  following 
modules  therefore  no  specific  uses  of  exercises,  case  studies  or  videos  are  required. 
Resources  Available:  time,  knowledge  and  attention,  starts  with  the  prime  decision 

maker  and  their  resources,  and  flows  outward  — >  other  crew  members  — >  aircraft  systems 

— >  ATC  — »  other  aircraft  — »  ATOC,  etc.,  all  crewmembers  have  external  resources  to 
draw  on.  Management  of  time,  attention  and  knowledge  are  all  related.  It  all  gets  down 
to  the  trade-offs. 

Managing  Time  and  Attention:  two  sides  of  the  same  coin,  making  sure  that  the  team 
locus  of  control  covers  all  the  critical  variables  (e.g.,  FFA:  first  fly-the-aircraft), 
managing  the  timeline. 

Managing  Knowledge:  your  internal  knowledge  state  shapes  the  decisions  you  make, 
quality  of  the  decision  depends  on  your  knowledge  state,  what  you  don’t  know  CAN  hurt 
you. 

Leadership/Followership  Issues:  management  doesn’t  just  happen  -  it  is  an  active 
process,  leadership,  followership  revisited,  all  can  provide  leadership  in  resource 
management. 

Summary: 

Knowledge: 

1.  What  are  the  available  resources? 

2.  What  is  being  managed? 

Skills:  NA. 

Assessment: 

1.  Core  knowledge. 
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Management  of  Control  and  Attention 

Introduction:  review  the  idea  of  a  hierarchical  system  with  control  switching  from  loop 
to  loop,  loops  not  attended  to  are  not  under  control,  need  to  sample  at  a  rate  determined 
by  how  quickly  things  can  change. 


Adaptation  and  Learning:  control  is  necessary  for  adaptation  (learning)  and  the 
building  of  situation  awareness. 

Locus  of  Control:  depth  and  breadth  of  processing,  depth  and  breadth  of  control  -  AIP 
revisited,  individual  and  team  locus  of  control,  backing  up,  critical  loops. 

Control  and  Arousal:  active  control  involves  incoming  information  to  the  brain  which 
drives  the  body’s  state  of  activation  and  arousal,  adaptation  to  constant  stimuli,  arousal 
decays  in  the  absence  of  stimuli. 


Vigilance:  passive  control,  monitoring  and  vigilance  effects,  attention  wanes  when  out- 
of-the-loop. 


Control  and  Automation:  who  is  in  control,  complacency,  trust,  in-the-loop  versus  out- 
of-the-loop,  timeline  management  versus  knowledge  management,  what  does  automation 
do  for  you,  automation  failures,  lack  of  good  mental  models. 

Leadership/Followership  Issues:  AC’s  responsibility  for  the  assignment  of  roles, 
acceptance  of  responsibility  that  goes  with  this  allocation,  all  play  an  active  role  in 
achieving  team  locus  of  control. 

Summary: 

Knowledge: 

1.  Control  means ‘attending  to’. 

2.  When  out-of-the-loop  you  are  not  gaining  knowledge. 

3.  Automation  traps. 
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Skills: 

1.  Ability  to  manage  the  loops  under  control  (depends  on  goals  and  responsibilities). 

2.  Ability  to  maintain  depth  (and  breadth)  of  processing  when  under  load  (AIP 
revisited). 

3.  Knowing  when  to  use,  and  using,  alerting  (attention  capture)  and  backing  up  (closing 
loops)  behaviours. 

Assessment: 

1.  Core  knowledge. 

2.  Demonstrates  control  of  all  critical  variables. 

3.  Demonstrates  the  use  of  alerting  and  backing  up  behaviours  as  appropriate. 
Resources: 


Exercises 
Case  Studies 


Videos 


1.  tba 
Positive 

1.  BA  Indonesia. 

2.  United  811,  Honolulu,  HI 

3.  United  232,  Sioux  City,  KA 

4.  Aloha  1712,  Maui,  HI. 

5.  Pan  Am  543,  Boston. 

Negative 

1.  United  173,  Portland,  OR 

2.  Eastern  40 1 ,  Florida  Everglades,  Miami  FL. 

3.  China  Air  012,  enroute  SF  to  Taipei. 

4.  Markair  3087,  Unalakeet. 

5.  Mohawk,  Albany,  NY. 

6.  Northwest  255  Detroit,  MI 

Positive 

1.  CC-130  crew  managing  locus  of  attention  and  control. 

Negative 

1.  tba 
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Management  of  Time  Pressure 
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Introduction:  review  of  the  IP  model  and  the  effects  of  time  pressure  on  performance, 
workload  and  errors. 

Coping  with  TP:  IP  strategies  (recognition  primed  decisions,  move  from  Knowledge- 
Rule-Skill  based  processing,  speed  accuracy  trade-off  and  errors  of  omission,  rules  of 
thumb),  task  loading  strategies  (prioritisation,  shedding  and  controlling  input,  delegation, 
need  for  common  mental  models,  role  of  briefing,  postponement,  pre-planning)  reduced 
locus  of  attention  (tunnelling). 
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Stressors:  low  arousal  states,  high  arousal  states,  time  pressure,  chronic  and  acute  stress, 
stressors,  effects  on  channel  capacity  and  locus  of  attention,  recognising  stress  in  yourself 
and  others,  what  to  do  about  it,  responsibilities  of  AC  and  crew. 

Leadership/Followership  Issues:  each  crewmember  has  the  responsibility  to  manage 
their  own  time  pressure  by  the  use  of  appropriate  strategies  and  calling  on  all  available 
resources,  be  sensitive  to  task  loadings  of  others  and  do  something  about  it  if  you  can. 

Summary: 

Knowledge: 

1.  Effects  of  time  pressure  on  performance  and  errors. 

2.  Strategies  for  coping  with  high  time  pressure. 

3.  Effects  of  stress  and  recognition  of  the  symptoms  of  stress. 

4.  Responsibilities  for  managing  stress. 

Skills: 

1 .  Ability  to  recognise  situations  of  excessive  time  stress. 

2.  Ability  to  manage  the  timeline  by 

—  prioritisation, 

—  delegation, 

—  postponement, 

—  pre-planning. 

Assessment: 

1.  Core  knowledge. 

2.  Recognises  the  symptoms  and  situations  associated  with  excessive  time  pressure. 

3.  Demonstrates  timeline  management  skills. 

Resources: 

Exercises  1.  Have  groups  work  through  a  scenario  emphasising 

elements  each  team  member  brings  to  the  situation  and 
issues  of  timelines  -  what  information  is  timely  when. 

Case  Studies  Positive 

1.  Pan  Am  543,  Boston. 

Negative 

1 .  MD- 1 80  incident,  Austin  TX. 

2.  Mohawk,  Albany,  NY. 

3.  Markair  3087,  Unalakeet. 

Videos  Positive 

1.  CC-130  crew  actively  managing  the  timeline. 

Negative 
1.  tba 
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Management  of  Knowledge 


Introduction:  review  how  the  mental  model  shapes  actions  and  that  decision  time  and 
time  pressure  depend  on  the  highest  levels  of  individual  skills,  quality  of  your  decisions 
depend  on  your  state  of  knowledge,  managing  knowledge  is  largely  managing  your  locus 
of  attention,  managing  attention  and  managing  knowledge  are  complementary. 

The  Mental  Model:  consists  both  of  the  long  term  declarative  knowledge  and  short  term 
situation  specific  knowledge,  individual  knowledge  and  shared  knowledge. 

Situation  Assessment:  Situation  awareness  depends  on  situation  assessment  which 
requires  the  active  control  of  relevant  loops,  active  control  is  more  effective  than  passive 
control,  depth  and  breadth  of  the  mental  model  (situation  awareness  is  not  a  single  entity 
that  you  either  have  or  don’t  have),  high  time  pressure  results  in  reduced  locus  of  control 
and  situation  awareness,  offload  the  housekeeping  duties  and  focus  on  the  most  important 
strategic  and  tactical  goals  depending  on  your  role.  Seamster’s  studies  that  show  that 
many  cases  of  ‘lack  of  assertiveness’  are  due  to  inadequate  situation  assessment. 

Decision  Making:  decision  making  and  the  knowledge  we  draw  on,  decision  biases 
(confirmation,  primacy,  recency,  expectancies,  weighting  of  a  priori  probabilities,  order 
of  presentation,  etc.)  and  how  you  ask  the  question  might  determine  the  information  you 
receive,  satisficing,  knowledge  management,  role  of  feedback  in  adaptation  and  learning. 
Team  Decision  Making  and  Team  Performance:  differences  between  team 
performance  and  team  decision  making  (in  a  hierarchical  team  not  everyone  needs  to 
participate  in  every  decision),  the  relevant  cumulative  knowledge  of  the  team  usually 
exceeds  that  of  any  one  individual  (synergy  versus  antagonistic),  when  this  knowledge  is 
shared  a  better  decision  is  often  made  as  one  is  drawing  on  the  pooled  knowledge  (c.f. 
two  people  pushing  a  heavy  load),  trade-off  between  the  timeliness  of  the  decision  and 
the  quality  of  that  decision,  styles  for  high  and  low  time  pressure  (“...what  do  you 
suggest?”  versus  ...”I  suggest...any  problems?”),  need  to  make  a  decision  (team  decision 
making  is  not  an  excuse  for  the  AC  not  to  make  a  decision),  need  to  manage  the  shared 
knowledge  (what,  when  and  how). 

Leadership/Followership  Issues:  pro-actively  seeking  and  offering  knowledge  when 
appropriate.  Ensuring  each  stage  of  the  AIP  model  is  exercised. 

Summary: 

Knowledge: 

1.  Situation  assessment  depends  on  the  locus  of  control  (AIP  again). 

2.  Decision  biases  and  what  to  do  about  them. 
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3.  Asking  questions  and  considering  the  options. 

4.  The  role  of  the  team  in  this  process. 

5.  Differences  between  teams  and  groups. 

Skills: 

1.  Situation  assessment,  managing  the  locus  of  control. 

2.  Ability  to  prioritise  and  focus  on  the  most  important  strategic  and  tactical  goals. 

3.  Ability  to  generate  and  evaluate  options. 

4.  Ability  to  process  at  all  stages  of  the  AIP  model. 

5.  Ability  to  use  all  available  resources. 

6.  Ability  to  take  input  and  make  an  appropriate  decision. 

Assessment: 

1.  Core  knowledge. 

2.  Seeks  all  appropriate  sources  of  knowledge  to  support  decision. 

3.  Tests  mental  models  against  all  appropriate  knowledge  sources. 

4.  Offers  knowledge  appropriate  to  the  level  of  decision  being  made. 

5.  Resolves  conflicting  data  (ACs). 

6.  Exercises  the  AIP  model. 

Resources: 


Exercises 


Case  Studies 


Videos 


1 .  Various  examples  demonstrating  decision  biases  (e.g., 
the  green  taxi4). 

2.  Systems  knowledge  exercise  of  the  type  used  in  the  AS 
Army  course5  (similar  to  the  survival  exercise  but  made 
operationally  relevant). 

3.  Set  up  a  situation  in  which  team  members  have 
different  pieces  of  the  puzzle,  some  conflicting  and 
some  congruent.  The  object  is  to  see  how  the 
knowledge  is  managed. 

Positive 

1.  United  232,  Sioux  City,  KA 

Negative 

1 .  HTT  s  CC- 1 30  example. 

2.  AA  965,  Cali,  Columbia. 

3.  Tiger  66,  Kuala  Lampur,  Malaysia. 

1.  tba 


I 


4  An  example  used  by  Tversky  and  Kahneman  to  demonstrate  that  human  decision  makers  do  not  use  prior 
probabilities  in  their  judgements  appropriate  to  a  Baysean  decision  making  model.  See  Tversky,  A.  and  Kahneman, 
D.  (1974).  Judgment  under  uncertainty:  Heuristics  and  biases.  Science ,  185,1124-1131. 

5  Bonner,  M.C.  (1996).  Crew  resource  management.  In,  Minutes  of  Technical  Panel  7,  Sub  Group  U,  of  The 
Technical  Cooperation  Program.  Monterey,  CA,  USA:  The  Technical  Cooperation  Program,  Washington,  DC, 
USA.  63-68 
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Communication 


Introduction:  review  of  the  functions  of  communication,  need  to  open  and  maintain 
communication  channels,  communication  is  more  than  the  words,  how  is  the  information 
coded,  draws  on  common  mental  models  (e.g..  Who’s  on  first?),  verbal  and  non-verbal 
communication. 

Communication  Loop:  Sender,  encoding,  receiver,  decoding,  feedback. 

Barriers  to  Communication:  physical,  cognitive,  phasiology,  interpersonal  (styles), 
organisational,  cultural. 

When,  Why  and  How:  when  and  why  do  you  need  to  build  common  mental  models, 
when  is  it  better  to  stay  quiet,  the  sterile  cockpit  (“...cheer  up...”,  “...off  you  go...”)  and 
when  to  speak  up,  a  Captain’s  brief,  monitoring  other’s  workload,  timing  of  input,  how  to 
open  communication  channels  and  maintain  them,  how  to  increase  your  loop  gain  (active 
listening,  levels  of  assertiveness,  advocacy,  body  language,  key  phases),  breaking  down 
the  barriers,  say  what  you  mean,  listen  beyond  the  words  —  listen  to  the  message,  famous 
last  words. 


What  was  said 

“...we  have  a  slight  problem...” 


“...we  would  like  to  expedite  our 
arrival...” 

“...please  amend  our  ETA  XYZ  from 
1300  to  1330Z...” 

“...Tower  this  is  ABC  —  tell  the 
aircraft  following  we  are  experiencing 
a  slight  windshear  on  finals...” 

“...Houston  we  have  a  problem...” 


What  was  meant 

“...we  have  an  uncontrollable  engine 
fire  and  the  main  spar  is  about  to 
fail...” 

“...we  are  10  miles  out  and  #4  has  just 
flamed  out  due  to  fuel  starvation...” 

“...we  have  no  idea  of  our  current 
position...” 

“...our  airspeed  is  fluctuating  +  and  - 
30  knots  and  we  are  showing  full 
deflection  on  the  glideslope...” 


Apollo  13 


Open  Loop  Communication:  must  receive  closure,  acknowledgements,  high  proficiency 
crews  are  characterised  by  a  low  proportion  of  open  loop  communications. 

Effective  Communication:  common  communication  errors,  promoting  effective 
communication. 

Leadership/Followership  Issues:  exercising  responsibility  for  opening  and  maintaining 
communication  channels. 
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Summary: 

Knowledge: 

1.  Roles  of  communication. 

2.  The  communication  loop. 

3.  Ways  of  opening  and  maintaining  communication  channels. 

4.  Barriers  to  communication  and  what  can  be  done  about  it. 

5.  How  to  increase  your  loop  gain. 

Skills: 

1.  Ability  to  establish  and  maintain  communication  channels. 

2.  Ability  to  communicate  clearly  and  concisely. 

Assessment: 

1.  Core  knowledge. 

2.  Sets  the  tone  for  open  communication  channels. 

3.  Maintains  information  flow. 

4.  Uses  language  appropriate  to  the  situation  (names  items,  doesn’t  use  slang  or  non¬ 
standard  terms). 

5.  Conveys  meaning  (loop  closure)  in  minimum  number  of  iterations. 

Resources: 

Exercises  1.  tba 

Case  Studies  Positive 

1.  BA  747,  Indonesia. 

2.  UK  Tornado  crew. 

Negative 

1.  Cheer  up. 

2.  Off  you  go. 

Positive 
1.  tba 
Negative 

1.  Who’s  on  first. 


Videos 


APPENDIX  2:  REPERTORY  GRID  PROCEDURE  FOR  ELICITING 
OBSERVABLE  CAPTAINCY  BEHAVIOURS 
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Using  RepGrid  to  Elicit  Captaincy  Constructs 


For  the  purposes  of  interpreting  these  instructions  note  that  the  Administrator  is  the  person 
administrating  the  elicitation  process,  while  the  Subject  is  the  person  providing  the  Captaincy 
constructs. 

There  are  three  phases  in  this  exercise. 


Phase  1 


General 

In  the  first  phase,  the  Administrator  will  assure  the  Subject  that  all  information  provided  in  this 
exercise  is  strictly  confidential.  For  example: 

Administrator:  “ Hello ,  (Subject’s  Name),  lam  (Administrator’s  Name).  I  would 
like  to  welcome  you  and  thank  you  for  participating  in  the  Captaincy  Constructs 
Elicitation  exercise.  In  this  exercise,  you  will  be  using  the  interactive  RepGrid 
computer  program  to  unravel  behaviours  that  you  identify  with  the  notion  of 
Captaincy.  I  will  guide  you  through  all  necessary  steps  in  order  for  you  to 
complete  this  process  on  the  computer.  I  assure  you  that  all  information  provided 
is  strictly  confidential.  Let  us  start  by  having  you  fill  out  Form  A  and  Form  B.” 

Form  A  consists  of  the  following  items  that  the  Subject  is  required  to 
provide  information  on: 

Name  (Optional  but  highly  recommended) 

Current  Position  (e.g.:  Aircraft  Commander) 

Rank  (e.g.:  Major) 

Date  of  Birth  (day,  month,  year) 

Flying  History 

Total  Hours 
Total  On  Type 

Number  of  hours  in  present  position 

Aircraft  Commander  (all  multi-crew  types) 

Co-pilot  (all  multi-crew  types) 

Completed  CRM  course 
Yes/No  (when) 

Refreshers  (when) 

Form  B  is  written  in  a  self  explanatory  format  and  guarantees  the  confidentiality  of  the 
information  given  by  the  Subject.  In  fact  the  Subject  will  take  this  Form  from  the  testing  venue 
at  the  end  of  the  exercise. 

Form  B  requires  the  Subject  to  reflect  upon,  and  choose  by  name  or  other  identifier,  a  total 
of  6  captains  that  they  have  flown  with.  The  Administrator  needs  only  to  ask  that  the  form  be 
completed,  and  to  answer  any  questions  the  Subject  might  have.  The  Administrator  does  not  see 
this  form!  For  example: 

Administrator:  “Now  that  you  have  completed  Form  A,  let  us  introduce  Form  B 
which  you  get  to  keep  at  the  end  of  this  exercise.  I  will  not  need  to  see  what  you 
have  written  on  it.  It  is  merely  a  tool  to  help  you  with  the  exercise. 
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Although,  Form  B  is  self  explanatory,  please  do  not  hesitate  to  ask  me  if  you  have 
any  questions  regarding  the  Form.  There  is  no  rush,  so  please  take  all  the  time 
you  need  for  completion .” 

While  the  Subject  is  working  on  Form  B,  it  is  a  good  time  for  the  Administrator  to  prepare  the 
RepGrid  computer  program,  instead  of  looking  at  what  the  Subject  is  doing.  This  way,  the 
Subject  will  be  tend  to  feel  less  hurried  and  can  concentrate  on  the  task  of  choosing  the  6 
captains.  However,  the  Administrator  may  choose  to  prepare  the  computer  program  before  the 
arrival  of  the  Subject.  If  so,  the  Administrator  will  merely  need  to  double  check  the  preparation 
process,  while  the  Subject  is  filling  out  forms  A  and  B. 


Preparing  the  RepGrid 

We  start  the  program  by  clicking  on  the  RepGrid  icon.  The  “Welcome  to  RepGrid”  window  will 
appear.  In  this  window,  it  reads  “Please  enter  your  name  and  a  session  description.”  The 
Administrator  will  need  to  type  in  the  Subject  number  in  the  “Name  and  description”  field  (e.g., 
Subjectl)  and  click  on  "Done".  Each  Subject  should  be  given  a  different  Subject  number. 


The  “Grids”  window  should  appear  next  The  Administrator  will  continue  by  clicking  on  “New 
Grid.” 
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A  small  window  will  appear  asking  for  a  file  name  to  save  the  session.  Here,  the  Administrator 
should  type  in  the  Subject  number  (e.g.  Subjectl),  and  click  on  “OK”. 
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The  “Subject?-Elicit”  window  will  then  appear.  The  blocked  field  anonymous  will  appear  in  the 
box  situated  below  the  Showing  elicitation  status  statement.  The  Administrator  should  now  type 
the  Subject  number  (e.g.,  Subjectl)  follow  by  clicking  on  “Done”. 
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The  blocked  field  context  will  appear  next  inside  the  box.  The  Administrator  continues  by 
typing  in  “to  explore  Identifiable  Captaincy  Behaviour  (ICB)’\  follow  by  clicking  on  “Done” 
three  times. 
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Next  to  appear  inside  the  box  is  the  blocked  field  New  element.  The  Administrator  should  type 
in  the  following  Elements,  one  at  a  time,  clicking  on  “More”  after  each  entry  (except  the  last 
element  —  Captain  G). 

Captain  A  (More) 

Captain  B  (More) 

Captain  C  (More) 

Captain  D  (More) 
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Captain  E  (More) 
Captain  F  (More) 
Captain  G  (Done) 


After  entering  the  last  element  (Captain  G),  the  Administrator  will  click  on  “Done”  to  return  to 
the  “Subject?-Elicit”  window.  The  Grid  is  now  ready  for  testing  the  Subject . 


PHASE  2 

When  the  Subject  signals  that  Form  B  has  been  completed,  the  Administrator  will  place  the 
computer  in  front  of  the  Subject,  and  inform  them  that  reference  should  always  be  made  to 
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Form  B  when  relating  Captain’s  names  to  the  Elements  A-G. 


Construct  Elicitation 

The  elicitation  process  will  be  presented  as  cycles.  Each  cycle  will  elicit  one  bi-polar  construct 
under  the  context  of  Identifiable  Captaincy  Behaviours. 

The  cycle  starts  by  clicking  on  “Triad”.  Three  elements  should  be  presented,  and  at  the  top  of 
the  window  it  should  read  “In  what  way  are  two  of  the  situations  alike  and  different  from  the 
third? 
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The  Subject  should  click  on  the  field  that  contains  the  identifier  (Captain  A,  Captain  B,  Captain 
C,  etc.)  of  the  Captain  that  they  consider  to  be  different  from  the  other  two  (they  should  refer  to 
Form  B  to  assist  them  in  this  task).  It  will  be  highlighted  to  indicate  which  of  the  three  has  been 
selected.  After  the  Subject  has  made  a  choice,  a  new  window  should  appear. 


In  this  new  window,  the  Subject  is  required  to  perform  a  series  of  tasks. 

•  First  the  Subject  will  be  prompted  to  enter  in  “The  difference  in  this  one”  field,  what  they 
think  is  the  difference  between  this  captain  and  the  other  pair,  in  term  of  an  identifiable 
captaincy  behaviour  (e.g.,  “Doesn’t  invite  input  from  his  crew”). 

•  Secondly,  the  Subject  should  double  click  on  the  “Similarity  between  these  two”  field, ,  and 
enter  in  this  field  what  they  think  is  the  similarity  between  this  pair  of  captains,  relative  to  the 
captain  that  is  different  (e.g.,  “incorporates  crew’s  inputs  to  their  decisions”).  Note  that  these 
same  and  difference  categories  are  by  nature  opposites. 

•  Finally,  the  Subject  will  adjust  the  position  of  the  elements  (the  Captains)  on  the  newly 
formed  bi-polar  scale,  so  as  to  reflect  best  the  Subject’s  judgement  of  where  each 
Captain  should  fit  on  the  scale  relative  to  the  others.  The  adjustment  is  done  by  selecting 
(clicking  and  holding)  the  element  to  be  repositioned  and  dragging  it  to  a  new  position  on  the 
scale. 

The  Subject  should  reposition  the  elements,  along  the  newly  formed  bi-polar  scale,  starting 
with  the  pair  of  captains  who  are  similar.  This  will  force  the  Subject  to  further 
differentiate  between  the  Captains  who  are  similar.  As  mentioned  before,  this  can  be  done 
by  clicking  and  holding  on  the  element  to  be  moved  and  dragging  it  to  a  new  position  on  the 
scale.  Now  the  remaining  Captains,  whose  labels  (Captain  A,  Captain  B,  etc.)  appear  on  the 
left  side  of  the  window,  are  positioned  on  the  scale.  The  Subject  should  click  and  hold  on 
these  elements,  one  at  a  time,  and  drag  them  to  a  position  along  the  scale  that  represents  the 
degree  to  which  their  behaviour  is  characterised  by  the  statements  at  the  end  of  the  scale.  The 
Subject  is  free  to  reposition  any  existing  elements  on  the  scale  at  any  time.  Note  that 
reference  should  always  be  made  to  Form  B  regarding  the  correspondence  between 
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When  all  the  elements  are  place,  the  Subject  can  continue  to  another  Triad  by  clicking  “Done” 
and  then  “Triad”.  A  new  cycle  of  the  elicitation  process  will  begin.  The  Administrator  should 
stay  with  the  Subject  for  at  least  three  cycles  of  the  process,  and  then  leave  the  Subject  alone  to 
complete  the  exercise.  The  elicitation  process  will  end  when  the  Subject  cannot  think  of  any 
further  way  of  making  distinctions  between  triads. 

The  subject  may  try  several  triads  to  see  if  any  grouping  elicits  further  ideas.  If  a  particular  triad 
doesn’t  trigger  any  thoughts  the  Subject  can  use  the  cancel  button  and  try  again.  When  the 
subject  has  exhausted  all  possibilities  they  should  call  the  Administrator  for  the  final  Phase  of  the 
exercise. 
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PHASE  3 


The  Administrator  will  rejoin  the  Subject  at  the  end  of  the  elicitation  process. 

The  Administrator  will  click  on  the  “To  constructs”  button,  if  it  is  showing,  to  obtain  a  listing  of 
the  bi-polar  constructs  elicited  at  phase  2.  Here,  the  Administrator  will  need  to  align  the  bi-polar 
constructs  in  the  same  direction,  starting  from  left  to  right,  the  least  desired  pole  to  the  highly 

desired  pole  (e.g..  Lazy  Industrious).  The  Administrator  can  polarise  the  constructs  by  using 
the  “Flip”  feature.  Clicking  on  the  Flip  icon  button  after  selecting  the  bi-polar  construct,  will 
reverse  the  polarity  for  this  construct. 

Furthermore,  clarification  notes  describing  the  concise  meaning  of  each  pair  of  constructs  should 
also  be  included.  The  Administrator  can  enter  and  add  clarification  text  by  double  clicking  on 
the  bi-polar  construct  and  editing  the  entry,  or  the  Administrator  can  add  clarification  notes  in  the 
space  provided  in  Form  C.  This  process  will  ensure  the  co-directional  property  and  descriptive 
content  accuracy  of  all  the  constructs  elicited. 

The  Subject  will  continue  to  rate  the  importance  of  each  bi-polar  construct  in  determining  their 
overall  perception  of  Captaincy,  using  a  9  point  scale.  These  ratings  should  be  noted  by  the 
Administrator  using  Form  C.  If  there  are  any  constructs  that  share  the  same  ratings,  the 
Administrator  should  further  require  the  Subjects  to  rank  these  equal  rated  constructs  by  their 
relative  importance  in  determining  identifiable  captaincy  behaviour. 

Finally,  the  Subject  will  rate  themselves  on  each  of  the  elicited  bi-polar  constructs,  using  a  nine 
point  bi-polar  scale.  This  is  to  determine  how  “Self’  is  being  judged  on  each  bi-polar  construct 
by  the  Subject.  The  Administrator  should  record  these  ratings  using  Form  D. 

When  this  is  done,  the  Administrator  should  debrief  the  Subject  on  the  exercise,  ensuring 
confidentiality,  and,  attend  to  questions  and  suggestions  from  the  Subject. 

The  Administrator  should  make  a  copy  of  the  elicited  file  on  a  separate  diskette. 

Annexes:  Forms  A,  B,  C  and  D 
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Annex  A 


Form  A:  Captaincy  Elicitation  Study 

Subject  Name  (Optional):  _ 

Subject  Number:  _ 

Position:  _ 

Rank:  _ 

Age  (day,  month,  year):  _ 

Flying  History: 

(a)  Total  Hours: 

(b)  Total  Hours  On  Type: 

(c)  Number  of  Hours  in  Present  Position: 

CRM  Course  Completed:  Yes/No 

If  yes,  date  of  completion:  _ 

CRM  Refresher  Course:  Yes/No 

If  yes,  date  of  course:  _ 
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Annex  B 


Form  B:  Captaincy  Elicitation  Study 

The  information  provided  by  you  in  this  form  is  strictly  confidential.  You  are  the  only  person 
that  will  know  its  content.  You  will  take  this  form  with  you  at  the  end  of  this  session.  We 
will  have  no  record  of  this  document. 

1.  Take  a  moment  to  reflect  upon  the  captains  whom  you  have  served  with  in  your  flying  career. 

2.  Reflect  on  and  choose  the  three  best  captains,  and  list  them  below  (use  real  names,  nicknames, 
initials,  etc.): 


3.  Reflect  on  and  choose  the  three  worst  captains,  and  list  them  below: 


4.  Rank  order  your  three  best  captains  below. 


Best  of  Best: 

(A) 

Second  Best: 

(B) 

Third  Best: 

(C) 

5.  Rank  order  your  three  worst  captains  below. 

Worst  of  Worst: 

(F) 

Second  Worst: 

(E) 

Third  Worst: 

(D) 

6.  Please  turn  the  page. 
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Form  B  (Continued):  Captaincy  Elicitation  Study 

7.  With  reference  to  the  ordering  of  your  Captains  as  they  appear  in  steps  4  and  5,  please 
rewrite  the  names  of  your  selections  in  the  corresponding  boxes  below: 


Captain  A 

(A) 

Captain 

(B) 

Captain 

(C) 

Captain  D 

(D) 

Captain  E 

(E) 

Captain  F 

(F) 

Captain  G 

(G)  The  “IDEAL”  captain 

Keep  this  page  handy  for  ready  reference  during  the  remaining  part  of  this  exercise. 

8.  When  you  have  completed  this  step,  please  notify  the  Administrator. 
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Annex  C 


Form  C:  Captaincy  Elicitation  Study 

Date:  _ 

Administrator:  _ 

Subject  Number:  _ 

Number  of  Constructs:  _ 


On  a  scale  of  1  to  9,  rank  order  the  importance  of  each  construct  in  determining  identifiable 
captaincy  behaviour. 


1 

least 

important 

2 

3  4 

quite 

important 

5 

fairly 

important 

6  7 

very 

important 

8  9 

most 

important 

Construct 

1  to  9  Rating 

Further 
ranking,  if  any 
two  or  more 
ratings  are 
equal 

i. 

2.  _ 

3.  _ 

4.  _ 

5.  _ 

6.  _ 

7.  _ 

8.  _ 

9.  _ 

10. 

11. 

12. 

13. 

14. 

15. 

16. 

17. 

18. 

19. 

20. 
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Annex  D 


Form  D:  Captaincy  Elicitation  Study 

Date:  _ 

Administrator:  _ 

Subject  Number:  _ 

Number  of  Constructs:  _ 

Rate  “Self’  on  each  of  the  elicited  bi-polar  constructs. 


1  2  3  4  5  6  7  8 

least 
desired 
pole 


Construct  1  to  9  “Self4 

Rating 

1.  _  _ 

2 _ _ 

3  _ _ 

4  _ _ 

5  _ _ 

6. _ _ _  _ 

7 _ _ 

8. _ _ _  _ 

9  _ _ 

10  _ _ 

11. _  _ 

12 _ _ 

13.  _ _  _ 

14 _ _ 

15.  _ _  _ 

16 _ _ 

17.  _  _ 

18.  _  _ 

19  _ _ 

20  _ _ 


9 

most 

desired 

pole 
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APPENDIX  3:  BEHAVIOURALLY  ANCHORED  RATING  SCALES  FOR 

HFDM  EVALUATION 
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Individual 

’  Did  each  crewmember  appear  calm  or  rushed? 

1  Did  each  crewmember  prioritise,  delegate,  delay,  shed,  buy  time  appropriately? 


Crew 

•  Was  a  balance  achieved  in  the  imposed  time  pressure  of  all  crewmembers? 


Knowledge  Management 


s 

*8 

.s 

•c 
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Knowledge  Management 


Individual 

•  Did  each  crewmember  attend  to  (control)  the  variables  within  their  domain  of  responsibility? 

♦  Did  each  crewmember  direct  attention  when  appropriate? 


CO 

0\ 


Crew 

•  Did  the  crew  ensure  that  goals  were  achieved  by  monitoring,  acknowledging,  cross-checking,  backing  up,  etc? 
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